Identifying duplicate songs, artists, and albums on Rap Genius: not so easy! Lyrics by Tom Lehman
0
Before you can prevent users from adding duplicate artists to Rap Genius, you have to be able to identify duplicate artists. This is harder than it sounds – you can’t just look for duplicate artist names because different users will invariably use different names for the same artist. For example, Lil' Wayne and Lil Wayne (i.e., with and without the apostrophe) both refer to the same person. So different names does not necessarily imply different artists. The question is how different do two names have to be before we can infer that they refer to different things? String “Essence”The apostrophe case gives us a start: two names must differ by more than just punctuation – specifically, they must differ by at least one letter or number – in order to refer to different things. How do we use this fact to help us detect duplicates? Instead of comparing two names directly, we will instead compare their essences – where a name’s essence is calculated by removing all its non-letters and non-numbers. Here’s a Ruby implementation of this idea: class String def essence strip. downcase. gsub(/[^a-z0-9]/, '') end endHere’s how this works: "Lil' Wayne".essence "Lil' Wayne" == "Lil Wayne" "Lil' Wayne".essence == "Lil Wayne".essence The names aren’t equal, but their essences are, so they refer to the same thing Improving EssenceDoes our implementation of essence completely capture the idea of “sameness”? Is it possible for two names to contain different alpha-numeric characters and yet still refer to the same thing? Anything is possible when you’re dealing with natural language! Here are some modifications to String#essence I made based on experimenting with real-world data: Puff Daddy & The Family refers to the same thing as Puff Daddy and The Family So essence should treat & and and the same. Similarly, essence should treat + and plus the same Nipsey Hu$$le should equal Nipsey Hussle Essence should treat $ and s the same (at least in the case of rap names..) The Hot Boyz should equal Hot Boyz Essence should ignore leading articles The Hot Boyz should equal The Hot Boys Essence should treat trailing z’s the same as trailing s’s Da Hot Boyz should equal The Hot Boyz Essence should treat Da (and Tha) like The. (Again, at least for rap names) The finished productHere’s a version of String#essence that accounts for these observations. Am I missing anything? class String def essence strip. downcase. gsub('&', 'and'). gsub('$', 's'). gsub('+', 'plus'). gsub(/\bda\b/i, 'the'). gsub(/([a-y\-])z\b/i, '\1s'). sub(/^(th[ea]|a|an)\s+/i, ''). gsub(/[^a-z0-9]/, '') end end(Note: Essence doesn’t help you at all in the Tupac / 2Pac case. For this you have to manually compile a list of alternate names for each artist! lol!)
About “Identifying duplicate songs, artists, and albums on Rap Genius: not so easy!” by Tom Lehman
Read the complete lyrics to "Identifying duplicate songs, artists, and albums on Rap Genius: not so easy!" by Tom Lehman. On Lyrks you can follow along with the full text, explore the artist's discography, and discover related songs. The track is often categorized under Non-Music, Code, News .
"Identifying duplicate songs, artists, and albums on Rap Genius: not so easy!" is performed by Tom Lehman. This page provides the full lyric text for fans who want to sing along, study the songwriting, or compare versions across releases. Lyrks organizes lyrics by artist and song slug so you can bookmark and share a stable URL.
Music lyrics help listeners connect with emotion, narrative, and rhythm in a track. Whether you are learning English, researching a favorite chorus, or preparing for karaoke, having accurate line breaks and section labels (verse, chorus, bridge) makes the experience easier. We link to the official artist profile on Lyrks where available, including biography snippets, top songs, and chart placements when we have that data.
If you enjoy "Identifying duplicate songs, artists, and albums on Rap Genius: not so easy!", explore more songs by Tom Lehman using the links below. Chart and trending pages on Lyrks highlight what listeners are searching for this week. For copyright or correction requests, see our DMCA and contact pages.
"Identifying duplicate songs, artists, and albums on Rap Genius: not so easy!" is credited to Tom Lehman. Songwriting credits may include additional writers listed on the release; check the credits section on this page for linked collaborators.
Album information for "Identifying duplicate songs, artists, and albums on Rap Genius: not so easy!" is not available in our database yet. You can still read the full lyrics on this page.
Visit the Tom Lehman artist page at /artist/tom-lehman for biography, popular tracks, and links to more lyric pages.
