The English alphabet didn't always go from A to Z. If you'd been learning your ABC's in the 19th century it's very likely that your alphabet would have had a 27th member: &.
The symbol, according to Keith Houston's charming and authoritative Shady Characters: The Secret Life of Punctuation, Symbols & Other Typographical Marks, appears in the historical record for the first time in some anonymous graffiti in the most famously ruined of ancient ruined cities: Pompeii. The exact date of its appearance is unknown, but the 79 AD eruption of Mount Vesuvius, which buried the city in volcanic ash, does, as Houston notes, "impose a rather hard upper limit on the possible range of dates."
That very first ampersand was a ligature—that is, a character consisting of two or more letters joined together. Its creator was joining the letters e and t, of the Latin word et, meaning "and."
The first known example of the word ampersand in English is from the late 18th century, but the word's origin lies in a linguistic tradition that dates to several centuries earlier. Starting in the late Middle Ages, single letters that also functioned as words—think I and A—were referred to as letters with the aid of the phrase per se, to clarify that it was the letter being referred to, and not the word. The letter I, for example, would be referred to with the phrase I per se, I, which means in Latin "I by itself (is the word) I." When the 27th quasi-letter & was referred to it was called & per se, and, meaning "& by itself (is the word) and." That read as "and per se and."
"And per se, and" eventually evolved into ampersand, the word we know and love today. & the rest is history.