I crunched the numbers on my household (two people) awhile back.
Once you amortize weekly things like laundry, vacuuming, shopping and washing the dog they come out to less than an hour a day. Cooking and cleaning up after cooking is also less than an hour. The real time sink is weekly tasks that take substantial amounts of time. There are basically two categories of those tasks, cleaning lots of things (e.g. making everything spotless because guests are coming) and property maintenance. These numbers are very household specific since the difference between "needs to be done" and "should be done" are very much based on personal preference.
Assuming a western lifestyle and a household size of 2(!!) 4hr/day of housework would probably require a high maintenance property (lots of hedges to trim and grass to cut) and/or lots of cooking from scratch and/or heating with wood (and splitting it yourself) and/or doing a lot of cleaning that is probably not strictly necessary.
A castle is stretching it but if it's a small castle, the landscaping isn't too opulent, you aren't a neat freak and have the money to spend on labor saving equipment it should be doable on 4hr/day.
Spoken like someone who's never done housekeeping. 4 hours a day is a pretty normal amount if you need to wash clothes, sweep, mop, do a family's worth of laundry, prepare food, clean dishes, etc.
Yeah I am also wondering where this 4hr per day statistic is coming from. Dinner for two usually takes no longer than 1 hr a day, dishes usually put in the dishwasher every other day, and we reserve Sundays for cleaning, laundry and taking out recyclables. I cannot fathom anyone taking 4 hrs every single day to do housework, unless you have some kind of aversion to a dirty clothes bin or dishes piling up in the sink or Mr. Clean holding a gun to your head.
That's enough time to keep an average castle with a garden up and running.