Why do search functions everywhere not return what I search for?

YouTube did something extremely bizarre with their search.

They limit the results to like 5-10 videos and then replace the rest of the results with irrelevant videos under the "People also watched" tab where the results are the combined list of random words from the search query and "From related searches" tab where it's just random videos that belong to the same category of videos. This means that unless you're extremely blessed by the algorithm your video has basically no chance to show up in the search results, even if is relevant!

The weirdest thing is that the old search isn't even gone! You can still get relevant results after clicking "Filters" and then "Videos" in the "Type" column.

Who is this feature even for and what was the motivation for doing it? From the perspective of an actual user it makes no sense.

My guess is that if you do user research you'll find that for the vast majority of people exact token matches are not what they actually need or appreciate.

People run searches on their phones with typos. They don't have a good mental model for how search works, or how best to use it. If they search for "can puppy eat pork" they're actually looking for results about if dogs can eat ham.

As an expert user of search this infuriates me too, especially when tricks like double quoting don't get me what I'm looking for.

But as someone who often implements search engines I'm spending a lot more time thinking about fuzzy matches and semantic search (things like searches that use vector embeddings generated by language models).

Elasticsearch or a functionally equivalent competitor is a straightforward commercial-off-the-shelf solution for text search. Default settings for it include fuzzy matching and returning "relevant" results instead of only exact ones.

Pretty sure there's nothing more nefarious going on than a bunch of different engineers and different companies plugging in COTS text search solutions with minimal customization.

Yes, and I see this phenomenon everywhere now. They completely ignore the explicit instructions we are giving and prioritize their “suggestions” instead. Facebook, Twitter with their default timeline views. On Twitch’s TV app it used to be one click from the main page to continue watching a video now it’s 5 clicks because they replace with suggested channels, recommended blah blah.

One can go blue in the face “quoting” terms on Amazon search but they’ll just show whatever they want anyways completely ignoring the exact terms entered. I’d prefer them to say “we couldn’t find any results for you” than to give me pages of useless results instead.

This topic (or a variation of it) comes up quite frequently on HN.

I agree with your observations. Google search seems to completely ignore search terms in quotation marks. It's like the search favours showing somewhat relevant results instead of nothing at all.

On Amazon, sorting by anything other than "Featured" is absolutely useless.

Instacart will show a couple of relevant items, and then a bunch absolutely random results, some of them being the opposite of what I asked for.

A lot of that is coming from optimizing for conversion. Turns out search isn't supposed to be accurate, it is supposed to generate money. You just happen to be the minority that cares about the exact thing.

Another reason is that search is just... hard.

Though in Amazon's case I bet it's just sheer incompetence caused by monopoly.

Yea it is truly awful.. e.g. job search has become even worse than it was a few years ago. The only good platform (Stackoverflow) has closed its job search. Most jobs seem to be on Linkedin these days.

However for a while now Linkedin job search seemingly shows me random stuff.. I can search for C++ and find all kinds of Java, Javascript, PHP jobs. It is driving me mad.

Any recommendations for job search sites (maybe even with negative filters)? I know about Who is Hiring? of course but there are not too many jobs for my region usually.

Yes, this drives me bonkers as well! The worst is when you want to sort on top of this, e.g. searching Amazon or eBay for "ThingA ThingB" and then sorting by lowest price first, you end up with pages of ThingX, ThingY, ThingZ with no mention or relevance at all for searched ThingA or ThingB. Quoting the terms occasionally helps but often does not.

I'm OK if you want to try returning "similar" results but there should be a way to 100% disable this, otherwise it makes your search borderline worthless.

It's to drive engagement. Why give you exactly what you want, when they can make you click some other stuff? There's no real competition, and if there were, the competition would soon realize it would have more revenue if it used the same strategy.

You can bet that somewhere, someone was looking at some A/B testing with these "features" and they saw an uplift of X% in revenue, so why not do it?

The current one that is driving me nuts is eBay's search. The "Advanced" search allows you to specify "From preferred locations" which it then completely ignores.

This makes eBay effectively unusable for me (I'm not up for paying excessive shipping and import duties combined with long shipping delays) so I end up using local auction sites instead.

On a related note Etsy still hasn't got the message that the UK isn't part of the EU any more, so they too pollute the searches with stuff that would require long delays (customs holdups are rife) and import duties.

I think this because these modern search functions use a data structure called "trie" for auto-completion. So when you want to search for a term and enter the first character, it looks like a tree-like structure to narrow down what you want to write.

I can never find what I want to search if I write the first letter incorrectly. And this happens a lot if you use a Turkish keyboard. In Turkish, the i letter is different from the I. There is also a capital I and a capital i. i cannot even search for instagram or ikea even on my phone without pain.

Apples App Store is the worst. You MUST have a UNIQUE app name on the store. But if you search for the EXACT app name, you DO NOT get the app! It's the most absurd situation you can imagine.

As someone who wrote a very silly and small search system with an elastic search backend.. this thing is a bit complicated.

If I do exact match for single words life is easy

Once you add phrases or punctuation and tell elastic search to find the best match (text analysis) then life isn't so simple anymore. It feels a bit like magic and I honestly cannot be bothered to become a language expert to know how to tune the machinery. The docs aren't great also.

Maybe there's a missing middle ?

Of course that if you're talking about social media, or search focused companies, they likely have other interests

The vast majority of users are not technical. They don't want keyword search, they want to find the information they're looking for. In that sense, they treat a search box more as a chatbot rather than a literal search tool. That is, they want something more like ChatGPT, not Google. That Google doesn't yet offer this functionality is incidental (it does sometimes and for limited use cases, such as surfacing weather details or lyrics, not pages with the literal words "X weather" or "Y lyrics").

I often find that too, that it doesn't work. I tried different search engines and they tend to have some advantages and some disadvantages, but overall, it does not help. (Although, some things might just be obscure and difficult to find anyways, possibly; and, sometimes, it is difficult to know what search terms to use. But, sometimes even if I know what is needed and put it in exactly, still it does not work.)

It could be that everyone just delegates to elastic search or similar which I think might be incapable of exact string searching (a trade off to get better fuzzy searching and speed). Tuning ES is probably possible but takes commitment and is complicated I suspect most don’t bother! I used it in one side project and a lot of tuning is needed for your use case.

I think search engines have realized that "searching" as a discovery method is slowly on its way out. Whether that's from SEO spam, AI-generated articles, undesirable (from the search engine's POV) sites displaying in results, etc. So, they want users to get used to search results really being "curated results."

Oh it’s the engineers who think they know better than the common man/woman. Elitist, snobbish etc.

Most people are not advanced searchers, so they expect fuzzy matches (give me what I mean vs what I ask for).

Exact searching can be expensive to add right and since most users don't really care, it's usually not worth it.

My theory is that search is, from a ux perspective, seen as a way to generate “browsing” opportunities rather than pinpoint something specific.

At the same time there’s a wish for searching for “what you meant” rather than “what you typed”.

Try search "white couple" in google images. Results are irrelevant in 50%.

https://www.google.com/search?q=white+couple&source=lnms&tbm...

However "black couple" results are more relevant.

https://www.google.com/search?q=black+couple&source=lnms&tbm...

It's hard to believe that results are not manipulated for purpose.

This explanation seems reasonable, however should be fixed. https://www.seroundtable.com/google-image-search-results-rac...

From my experience implementing search using Solr (or Elasticsearch) is simple to get the basics and hard to get really really good. So I guess most developers/companies are satisfied with "good enough".

Because search results are super competitive and they bombard SE with highly optimized tactics to rank higher even for irrelevant terms.

As as suggestion, try searching (or asking for recommendations) on ChatGPT for the same keyword

I agree. I actually want literal search most of the time. Not the smart kind.

It happens even in Google Scholar. A few days ago I searched for 'influenza' and some other term and got 80% SARS-COV-2 results. And not the paper I searched for, although it is a 2022 paper.

Normies. It is because of normies who are too dumb to use a computer. They couldn't work a boolean search even in english to another person so search has to be ruined for the rest of us.

And it keeps getting worse!

No solutions, but I empathize and it drives me nuts too.

Ha! Slightly tangent—I do payroll using ADP’s Run web product. Searching (filtering) the employee list is bonkers.

Take a name like, “Noel”.

If you type “Noel”, you get nothing.

Type “No”, you get “Noel”!

The simple reason is that these search engines don't exist to help you find information.

The search engines exist to try and sell you stuff.

What do these companies use internally, when they actually have to find something in their own inventory for themselves?

I guess you know the answer, and it has been mentioned several times already: search is "bad" from the seller's point of view, they don't want you to buy what you want. Instead they want you to buy what they want to sell, and that is of course easier if they control the presentation of products. Search is an explicit way of trying to give control over what is shown to the user, and that runs counter to the underlying optimization for maximum sales.

At least I guess so, the nearest I have been to any kind of sales optimization pipeline is buying a lot of stuff online. :)

Smaller and niche sites can still do Real Search, but it's the large marketplaces with thousands of everything (where, you know, you really want good search) that usually instead favor their own optimization.

Actually, but perhaps surprisingly, a site like Aliexpress actually kind of explain this in their search help text, which is at least trying to be transparent. I like that, but it still drives me nuts when the search results change when you change to sort by price. That's like ... "impossible" in my mind; changing the presentation order of a list should not change the contents o of the list, but there you are.

Aargh, basically. I wish it could stop.

Applying Standard Capitalist Rules, I guess in the future you'll need to pay a fee to each site for the right to search and sort exactly and disable the optimizations for you. Fun times.

EDIT: Minor grammar fix.

In Google put your phrase in quotes and it’ll find the exact word or words

Yes, it's terrible, happens all the time

Greed