Data Analytics and BI

Data analytics usually stalls not on tooling but on the fact that three departments count the same revenue three different ways. So before dashboards I get the definitions and the sources onto common ground: otherwise a report just spreads inconsistent numbers faster.

What people arrive with

The board asks for last quarter’s revenue. Finance brings one number, sales another, operations a third, and each of them is counted honestly: one after returns, one on shipment, one on the payment date. Half a day goes on arguing whose figure is right, and the decision moves out by a week.

The dashboard built a year ago takes no part in that conversation: it showed a fourth number, and people stopped opening it. Clients reach me at different points in that story, but the work never starts with a platform: first I trace where each number comes from and where the rules parted company.

Data analytics starts with definitions

“Revenue” is with tax or without, on shipment or on payment, before returns or after them, in dirhams or in the contract currency, and at the rate on which date. Until there is one written answer, every data mart stays arguable. So the first deliverable is a metric glossary that finance, sales and operations have all signed off.

• The formula in words and in SQL, so it cannot be read two ways.
• The name of the person people go to when finance and sales bring two different numbers: without one, the argument moves into email and stays there.
• Source and refresh time: which system it comes from and as of when it is current.
• What the metric does not show, written down separately; it prevents wrong conclusions.

Warehouse and data flows

Next the data has to land in one place where it can be joined. That is separate engineering work, taking the larger part of the project: some sources have no API, others hand over scheduled exports, others live in spreadsheets on laptops.

• An inventory of sources: what sits where, who owns the system, what volume it holds and how often it changes.
• Scheduled loads with a log of exactly what was loaded; without it, a discrepancy cannot be explained.
• A transformation layer where raw tables become data marts built on the agreed definitions.
• Quality checks at every step (empty keys, duplicates, a load that stopped halfway). The message goes to the owner of that data mart before the morning stand-up, and failures show on the Grafana board over Prometheus metrics.

Load is worth planning for in advance. On Monolith Plus the reporting queries competed with the database that took the operations flow, so the analytics side had to move onto a copy that lags behind. The fork shows up in every project: a copy and stale numbers, or the live database and a risk to its response time.

Dashboards people actually open

A dashboard differs from a report in that people look at it voluntarily. That happens when the first screen holds five to seven numbers, each answering a question the reader already asks every morning, and any of them opens down to the rows behind it.

So the layout is agreed before anything is built: I sketch the screen, show it to the person who will use it, and listen to what they intend to do with it. Some charts fall away at that point, which is a good outcome: every extra chart costs maintenance time.

Forecasts and anomalies

A forecasting model earns its place when there are a couple of years of history and a seasonality that repeats. With less history, or a business that changed its product range three times in that period, the honest answer is a moving average plus worse / expected / better scenarios.

Early on, spotting deviations is more useful than forecasting: an order that falls outside a client’s usual profile, a payment outside working hours. My six patents are in information security and Big Data: the problems there are very close to these.

Stack

The toolset is chosen around what the company already runs: the fewer new names on the list, the sooner your analyst starts editing the data marts on their own.

Layer	Tools
Warehouse	PostgreSQL, BigQuery, Snowflake
Loading and transformation	ETL / ELT (Airbyte, Fivetran), dbt, Apache Kafka
Reports and dashboards	Microsoft Power BI, Tableau, Looker Studio
Calculation and models	Python, SQL

What I do not do

• I do not build dashboards before the metric definitions are agreed: otherwise the argument about numbers moves to a new screen.
• I do not promise a forecasting model where there is no history for one: what goes into the report then is a moving average and three scenarios.
• I do not shape the toolset around a vendor’s price list: licences are issued to your company directly, and the invoice for them goes to you, not to me.
• I do not leave behind a system nobody can run: if no one in the company is accountable for data, I suggest starting with that person rather than with a warehouse.
• Read access inside the warehouse I do configure, but encryption, audit logs and the personal data policy are not mine to take on: that is information security.

How the engagement runs

I start with a single number rather than with the sources. You name the figure that gets argued about most, and I follow it from the board slide down to the row in the accounting system. That takes one to two weeks and ends in a diagram: where the number is born, at which step it changes, and which version to trust until it is fixed.

After that the work runs in stages: warehouse and loading, glossary and data marts, dashboards, handover. Every stage ends in a load that runs, a data mart or a screen you can open. A full project typically takes eight to twelve weeks and costs $20,000 (AED 73,400) as a fixed project price, with no hourly billing and no open-ended scope.

The first review of what you already have ends in a single page: where your reports diverge and which discrepancy to close first. Sometimes the report turns out to be a matter of configuring an export in a system you already pay for, and then there is no project.

Related services

If data is being moved between systems by hand, the next step is business automation. A warehouse holding client data ends up inside the perimeter: that is information security. For a two or three year horizon the numbers pair with strategic planning, and for management reporting and budgets there is financial management.

Frequently Asked Questions

How soon will the first working dashboard appear?

Usually in week three or four, once loading from the main source is in place and the first metrics are agreed. It can be done sooner, but the result is a picture nobody trusts.

What happens when the numbers diverge again?

They will diverge: systems close their periods at different times. So every data mart carries a reconciliation against its source and a mark of what it is current as of, and past the agreed threshold the load notifies the metric owner: the conversation is then about specific rows.

How much time will the project take from your team?

Most of it goes not on the technology but on agreeing definitions: sessions with finance, sales and operations where a disputed metric is taken down to a formula.

Who adds new metrics after the project?

Your analyst or developer: the SQL behind every data mart sits in your repository next to the glossary, and a new metric is made by copying the nearest one and editing the formula.

---

Ready to Get Started?

How many versions of last quarter’s revenue are circulating in your inbox? If it is more than one, name that metric on the first call: I will follow it down to the row in the accounting system. Book a conversation.