ETYMOLOGICE

There are a lot of English words ending in -ce. In most cases (think convince, advance, lenience​) they just come from Latin, but a small minority come from Old English and Proto-Germanic. Pretty much all of these Germanic words used to end in an s, but the spelling was changed to preserve the breathy sound of the sibilant. One well known example is the plural of die being dice, although it was originally dys. Here are some others:

• Bodice is a plural of body because the garment came in two parts that went on either side of the body
  
• Twice comes from Old English twies, which also meant "twice"
  
• Once comes from Old English anes, the genitive of the word for "one"
• Pence was originally a contraction of penies, an old spelling for "pennies"
  
• Deuce, although originally French, was influenced by Germanic words to change its plural from dews
  
• Hence comes from Old English hennes, with the -s indicating an adverbial genitive
• Since replaced synnes in the sixteenth century
  

I thought that was pretty cool!