Cleaning the noise, not the language

I needed Arabic reviews. A lot of them.

I was working on sentiment analysis and wanted to do it against the way people actually write, not against a few hundred tidy examples or something synthetic. The Arabic datasets I could find were either too small for what I had in mind, or labelled in a way that did not fit the problem.

There were two ways out of that. Change the problem to fit the data that existed, or build the data. I built the data.

Three sites, three scrapers

The reviews came from Amazon, Noon and Jumia.

I wrote the scrapers myself, which meant opening each site and reading how its pages were actually put together before I could pull anything out of them. I had assumed I would write one scraper and point it at three sites. That is not how it went. Each site had structured its HTML differently enough that nothing transferred, and every source ended up being its own problem, solved separately.

That is worth knowing before you start, because it changes the estimate. Collecting from three sources is not one job done three times. It is three jobs.

The raw data was worse than untidy

I expected typos and missing fields. What was actually there was more interesting than that.

The same sentiment arrived in a dozen different shapes. There were diacritics. There were emoji. There were stretched words — حلوووو, with as many و as the writer felt like. There was Franco-Arabic, Arabic written in Latin letters and digits, where 7abibi is حبيبي.

The obvious move is to call all of that noise and strip it. I nearly did.

The reason I did not is that some of it carries the meaning. Someone who writes حلوووو with four و is not saying the same thing as someone who writes حلو. The stretch is the feeling. Delete it and you have a tidier-looking row that has lost the signal you were trying to learn from.

So the question stopped being how do I clean this and became what here is noise, and what is the language doing its job.

The rule: clean the noise, not the language

That is the sentence I ended up working to, and it settled most of the rest.

Diacritics get handled or removed depending on what the pipeline downstream needs. Emoji and elongation stay, because they carry sentiment. Franco-Arabic stays, because throwing away a review for not being in Arabic script is throwing away a real way that real people write.

Dialects stay separate. Egyptian stays Egyptian, Gulf stays Gulf, Levantine stays Levantine, Modern Standard stays Modern Standard. The pull toward normalising everything into MSA is strong, because it makes the dataset look tidier. It also makes it represent a language nobody is writing in the reviews.

Over-normalise and you get a dataset that is cleaner on paper and a worse description of the thing your model has to handle.

I did remove one category outright: profanity and insults. Not because they were noise, but because I could not handle them reliably, and shipping something I could not stand behind seemed worse than leaving it out. That is a decision rather than a fact about the data — a distinction I will come back to.

Stars beat words

I used the star rating as the primary sentiment signal rather than the text.

That sounds like the lazy option until you meet sarcasm.

Someone writes المنتج تحفههههه — the product is a masterpiece — adds laughing emoji, and gives it one star. Read the text alone and it is glowing. Read the stars and it is the opposite. The words are doing something the words do not say.

Arabic reviews are full of this: praise vocabulary used to deliver a complaint. Label from the text and you will label those wrong, confidently, at scale.

The star rating is not a perfect oracle either. People misclick, and people rate the delivery instead of the product. But it is an independent signal, produced by the same person at the same moment, and it disagrees with the text exactly where the text is least trustworthy.

What it does not represent

The dataset ended up at around 330,000 reviews, labelled positive and negative.

If you use it, the thing to be careful about is treating it as a description of Arabic in general. Several dialects being present does not mean they are present in equal proportion. The sources are three e-commerce sites, which is a specific kind of writing, by a specific kind of writer, about a specific kind of thing. That is a bias, and it is not a defect — it is what the data is.

Before training or evaluating on it, look at the distribution and the sources. Whatever comes out is a statement about product reviews on those platforms, not about the language.

For any dataset, you need to know what it represents. More importantly, what it does not.

What I would do differently

I would document the decisions while making them.

Every ambiguous case — sarcasm, mixed sentiment, Franco-Arabic, a dialect I was unsure how to treat — got resolved somehow while I was working. What I did not do was write down the rule I had just invented, and why. At a few hundred rows that does not matter. At 330,000, a small choice made early is applied to an enormous number of records, and months later you cannot reconstruct what you decided or whether you stayed consistent.

Next time I would write the guidelines first and let them grow as the odd cases arrive, so the reasoning survives alongside the data.

The part that surprised me

I thought the hard part would be reaching a big number.

It was not. Collecting 330,000 reviews is a scraping problem, and scraping problems are annoying but tractable. The hard part was everything after: dialects, writing styles, sarcasm, Franco-Arabic, emoji, elongation, and deciding for each of them where the line between signal and noise sits.

That changed how I think about datasets. A dataset is not a pile of text. It is an engineered artefact with decisions, assumptions and limits built into it — and the decisions you make while building it are as much a part of it as the rows.