What Genre is My Book???
If you're writing fiction, you need to nail down your genre. Even if you're self-publishing, you need a way to market your books. There is absolutely not one single book in the whole world that appeals to everyone, and genres help you find your audience.
Category vs Genre
Lots of people think of Young Adult (YA) or Middle Grade (MG) as genres, but those are categories. I'll likely go into categories in a different post, but for the purposes of this post let's think of them this way: Categories determine the age of your audience, genres are determined by the preferences and personalities of your audience (or your characters).
The "Literary" Genres
Literary Fiction - This is sometimes referred to as "general fiction" or even just "fiction". This is fiction that doesn't fit neatly into one of the commercial genres. It's usually set in contemporary times or an ambiguous time, and it is almost always free from any supernatural elements. Occasionally, you'll get some minor mystical elements or some unexplainable phenomenon, but rarely. Literary fiction focuses on prose, on how pretty the words sound and how they elicit feelings in the reader. Character, plot, and setting still matter, of course, but they aren't the shining star.
Women's Fiction - This is literary fiction written by women. Almost always has a female protagonist and will deal with "women's issues" more often. It's kind of a bullshit genre, because women's stories are human stories and shouldn't be shoved to the side. Though, I suppose, if you really don't want to read a story by a man, it's a good filter to have.
Upmarket/Book Club Fiction - This is literary fiction that feels more commercial. The term "upmarket" is an industry term, but "book club fiction" makes it pretty clear what type of books we're talking about. It still checks all the boxes of literary fiction, but it's maybe a little faster paced, a little more voicey, a little less esoteric.
Magical Realism - This is sometimes misunderstood as "light contemporary fantasy" but that definition is woefully inadequate. Magical Realism is literary fiction that has a tiny element of the mystical or supernatural in it that completely alters the path of the story, but it is not a fantasy story. Some literary scholars argue that only Latin authors can write Magical Realism, others say that anyone born into a culture that suffered at the hands of colonialism can write it. Suffice it to say, I am neither of those things so I will not wade into the waters of whether or not you should be writing magical realism.
Here's Tor's take on the difference between magical realism and fantasy: Not Fantasy, Sorry.
Here's a fantastic piece about the genre's origins, limitations, and the exploitation of it: What We Talk About When We Talk About Magical Realism
The "Commercial" Genres
Commercial genres are driven by tropes and conventions more so than literary fiction. Readers of these genres pick them up specifically for the tropes and will be very upset with you if you try to pull a fast one on them. These genres also all break down into nearly endless subgenres.
Romance - A story with a romance as the main plot. Always ends with either a "happily ever after" or a "happy for now" (HEA and HFN, respectively). There is an entire spectrum of sexiness/explicit content, ranging from "sweet" romances in which there is nary more than a closed-lip kiss at the end of the book all the way up to graphic on-the-page sex five or six times in a 250-page novel. Romance novels sell well; I've seen reports that romance publishing alone is worth upwards of a billion dollars every year.
If your story has a strong romance but does not end with a HFN/HEA, then you don't have a romance. You have some other genre with a strong romantic storyline. "Fantasy with a strong romantic storyline" or "Thriller with a strong romantic storyline" is how you pitch those stories. Do not try to sell a non-romance to a romance reader. They will be unhappy with you.
Some subgenres: romantic comedy, romantic suspense, sports romance, historical romance, contemporary romance, erotica, paranormal romance.
Historical - Pretty self-explanatory: It's a story set in the past. How far in the past? Depends on who you're writing for and why you're setting the story in that time period. If you're writing for teens, but you don't know what it's like to be a teen today so you set your YA novel in your high school years? NOPE. That's just lazy. If you're writing about being a teen during the rise of school shootings with the hindsight and depth of knowledge we have about these events today? Yeah, okay, you can write a YA historical set in the 90s.
The historical setting is a deliberate choice, a setting that makes the story work. The most popular historical settings are easily The Tudor Era, The Regency, and WWII, though you can find historical fiction set thousands of years ago on every continent and in every culture.
Some subgenres: Historical romance, fictionalized stories surrounding major real-life historical events and persons, war stories, westerns
Thriller - Thrillers are fast-paced and twisty, usually on the shorter side. Historically speaking, these have been male-driven stories with male protagonists and a lot of misogyny and violence on the page. This stereotype is shifting, though, and we're seeing not only women writing thrillers but men writing thrillers that appeal to a more modern audience, too.
Some subgenres: crime thrillers, spy/espionage thrillers, domestic suspense, courtroom thrillers
Mystery - There's been a crime (almost always a murder) and someone has to solve the crime. This genre relies on a satisfactory resolution and that means you explicitly reveal Whodunnit in the end. You can have a mystery buried within your story and have it not be exclusively a mystery (lots of stories have a mystery to unravel) but for it to be categorized as a genre mystery novel, you need to introduce the mystery at the beginning, spend the bulk of your pages solving that mystery, and have a satisfactory resolution, including some form of justice or comeuppance for the perpetrator.
Some subgenres: cozy mystery, detective stories/procedurals
Mystery and Thriller often share a lot of the same readership and are sometimes lumped together as "mystery/crime fiction".
The "Speculative" Genres
These genres have at least one major element that doesn't exist in the real world. Mythical creatures like dragons or chimeras, magic, time travel, ghosts, paranormal humanoids like zombies or vampires or shifters, technology that doesn't yet exist, or coll technology that never existed.
Fantasy - Magic. That's it. That's all it takes to be a fantasy. It can be complex like a Brandon Sanderson novel, or it can be subtle like in A SONG OF ICE AND FIRE/GAME OF THRONES (aside from some magical creatures and the fact that the weather is wonky, the first book has no magic at all). It can be set in our world like TWILIGHT or it can be in a second world like LORD OF THE RINGS. Mythical creatures, magic systems, unexplainable phenomena.
Some subgenres: High fantasy, dark fantasy, epic fantasy, urban fantasy, paranormal fantasy, contemporary fantasy, fairy tales, myths
Science Fiction - Where fantasy uses magic to explain the unexplainable, science fiction tries to use science to predict how we could explain the unexplainable. Soft science fiction is something accessible, something most people can understand or be curious about. Back to the Future is a pretty accessible science fiction story, as is Jurassic Park. The science itself isn't really the point, and scientists tend to get pretty annoyed at these stories. Hard science fiction is the other end of the spectrum. The story relies on reputable scientific theory and often delves into the details of said theory at length within the story.
Some subgenres: space opera, near-future, historical science fiction, space travel, time travel, biogenetics, dystopian/post-apocalyptic, apocalyptic/disaster stories
Horror - The supernatural element in the story is dark, menacing, and scary. It can be violent, but isn't necessarily so. Ghosts and zombies and vampires, sure, but also it could sometimes just be a dude with a really big knife. Sometimes there's zero violence on the page and the horror is all psychological.
Some subgenres: body horror, fantasy horror, apocalyptic, gothic horror
The -punks - Steampunk is probably the most popular, but cyperpunk, dieselpunk, hopepunk, solarpunk, climatepunk, and a whole host of other -punks are out there. These are most simply described as "historical science fiction" or might even be an alternate reality. The "punk" on the end tells us that there's a twist: Steampunk takes the era of the steam engine (mid-late 1800s) and extrapolates what science fiction in that era would look like (I used to describe this as "corsets + robots"). Dieselpunk is set during the era in which gas engines were taking over the world (early 20th century). Cyberpunk is an alternate, sci-fi version of our current cyber-connected world (like The Matrix).
All of these speculative genres can overlap by a large margin. Where do superhero stories fit? It's functionally magic, but usually with a pseudo-scientific explanation of how it all got started. What about dystopian fiction? It's set in the future, so the technology is advanced and intriguing, but everything has gone horribly wrong and death waits around every corner.
If you're pitching your work to agents and editors, you can get away with maybe mashing up two genres: historical romance, literary thriller, dystopian fantasy. But if you're trying for a third, you are going to run into some trouble. Nobody knows how to sell a historical thriller romance horror. If that really, truly sounds like your book, then you need to focus in on two genres and mention the strong elements: fast-paced historical horror with a romantic subplot. Or perhaps: historical romance with supernatural elements and a police procedural structure.
Pitching too many genres all at once is confusing and shows that you 1) don't know the market and 2) are trying to do too much or 3) don't fully understand the marketing purposes for these genres to exist.
Pick one genre, maybe give yourself a modifier for that genre and then relegate the rest of your story to the blurb and synopsis.