Sources conflict as to whether he was born in 1755 or 1757. If was born in 1755, he would be 18 in his episode (he was visited in 1773), If he was born in 1757, he would be 16.
That would mean the second time they visit him (Berby fast-forwards "twenty-one years later") in 1794 that he would be 37 or 39 years old.
18 has been the age of majority (for voting) in the United States since 1971 when the 26th Amendment was passed and ratified, preventing higher ages from being set. Prior to this (such as in the 18th and 19th centuries) the age required to vote was 21 years old, so that would've still been considered a kid in their own time, if not modern day.