Maybe you read Daniel Keyes' short story "Flowers For Algernon" in high school, or saw one of the tv shows or movies based on it. The original short story won a Hugo award in 1960. Certainly a classic.
But I bet you've never read the story that came in second in Hugo voting that year- Philip José Farmer's "The Alley Man". I came across it more or less by accident as it was in one of the many short story anthologies I buy for a buck in used bookstores that serve me so well on nightshift at the coal mine.
The alley man is Old Man Paley, the last surviving Neanderthal, a one-armed bottlepicker who lives off the refuse of us False Folks, worshipping the Old Guy in the Sky and patiently waiting for the time when he can rise again.
Or maybe he's just an ugly and demented old derelict whose biggest dream is to be a wrestler on tv, billed as the Awful Apeman.
Anyways, a great find. You should read it- the real pleasure is listening to Old Man's constant stream of chatter as he recounts his people's history to a wide-eyed anthropology student. Obviously I ate it right up since it was about Neanderthals. Or probably was about Neanderthals.
Speaking of Neanderlit, I recommend Dave Wolverton's Serpent Catch and Path of the Hero, which has Neanderthals and mammoths and all kinds of crazy things.
I'm sure you've at least heard of, if not read, Clan of the Cave Bear. Good book, could have used more actual cave bears though. Still, it has plenty of Neanderthals.