It’s cricket.  The bat on the right appears to taper and swell from the handle to the end, as cricket bats did in the late 1700s when they were transitioning from the long, narrow “hockey stick” type to the modern straight, wide bat.  The widening of the bat in this image seems to go downward accordingly, in one direction, like a butter knife is flat along the top surface but curved out on the bottom.  Bats were shaped like that at the time to swing at balls that tended to stay lower to the ground.  I think I see stitching on the ball that resembles a modern cricket ball,and the ball is being held as it was bowled in cricket at the time: underhanded.  The subject of the portrait is two batsmen (partners) with their own bats, like in cricket, as opposed to baseball where only one batsman is the focus. 

But more significantly is the context of the times.  Cricket was THE most popular sport in America and Britain at the time.  Sure there were sprinkles of references to other stick and ball games but cricket was far and away the most well known.  For example, in the colony of Georgia, on the periphery of English settlement in North America, cricket is the only sport mentioned by name, and it is mentioned often in the historical record, even in the early national period.  Savannah had a cricket club up to the start of the Civil War.  Moreover, there must be dozens if not hundreds of portraits in existence that date from that era in which boys are posing holding cricket bats.  See, for example Francis Cotes, “Charles Collyer, as a boy, with a cricket bat”, 1766. 

On the topic of the multi-racial nature of cricket, here is a link that may also be of interest: 

http://en.wikipedia.org/wiki/Barbados_Cricket_Buckle

It concerns a buckle with an image of a slave playing cricket in the English colony of Barbados around the same time that these two boys, who appear to be of different races, became the subject of this portrait.