Then he operated as a sole trader, but used you and your brother as unpaid labour. Probably just about legal, but certainly exploitative of you.
Wasn't clear you had other jobs as you said without inheritance that two decades would have been "squandered" so made it sound like working for your folks was literally all you did.