(1 of 6) CHRT.com - Intro – Aaron Max Carver

Project Intro

I built CHRT with Kyle Reagan
- We’re colleagues from McKinsey and have been building software together for a couple years
At the end of August 2024, Kyle and I talked about how the future of analytics was going to be:
- much bigger than the present
- dominated by AI agents
So we built a product that does agentic analytics - some parts of it are described in this series of posts
But we aren’t working on CHRT / AI Analytics anymore. We’re doing something else now.

Post-Mortem

Future of analytics

The two coolest things in analytics that I can think of right (Nov 30, 2024) now are:
- Julius AI - like CHRT, but better. “Chat with your files and get expert-level insights in seconds.”
- MLE-bench - a reasoning model was given a laptop and GPU for 24 hours and tasked with Kaggle ML competitions - it medalled on 17% of them. So yeah, that’s the future
Before we built the product, I wrote a brief essay about the future of analytics.
- I expect that the two resources above will very much be mainstream before 2030.

Towards Full Self-Driving (FSD) Analytics

The Thesis

FSD Analytics will be a very useful thing.

It will do analytics for you. You will describe what you want, and it will generate it.

Actually - that’s what Autopilot for Analytics will do. FSD Analytics will offer much more. FSD Analytics will ask important, interesting, and insightful questions. Then, it will do the analytics, evaluate them, and discuss the findings with you.

Soon¹, FSD Analytics will outperform the median highly-talented person and thus will obviously dominate the market.

Does FSD Analytics mean no humans “in the loop”?

Humans will definetely still be in the loop.

To illustrate this point, here’s a question with a false premise: Do FSD cars eliminate the need for humans in cars?

Of course not!

The false premise is assuming that the only reason a human would be in a car is to move the car - like in long-haul trucking. But many billions of miles of driving is done for the purpose of moving the human, so the FSD involved there has no change on the need for having a human in the vehicle. They just won’t need to stare out the windshield and operate the power and steeting.

Similarly, many billions of analytics charts and report are generated each year for the purpose of informing humans. They’ll be able to get informed by data without needing to stare at the screen and control the keyboard.

Is FSD BI the same as AGI?

FSD BI is very different from AGI. I say that confidently, even without a consensus definition of AGI.

Let’s belabor the cars analogy: FSD vehicles are not AGI. They can drive to the store and back on their own, but they clearly can’t do “anything a human could”, or other similar definitions of AGI.

FSD BI will be very useful, but is not AGI².

Will AGI or ASI kill the need for analytics?

I think AGI will drastically increase the amount of analytics that is produced and consumed. I think it’s a Jevon’s Paradox situation - efficiency driving affordability driving more feasible use cases driving increased demand.

There’s certainly a big question about who will produce those analytics - the ~5 entities building trillion-dollar computer clusters to train huge models, or smaller and scrappier companies building product for their users.

If the Scaling Laws (or at least some scaling law) continue indefinetely, I’d bet everything on the single most well-resourced frontier lab (or the top few) that operate(s) under regulations which do not limit matrix multiplication.

If improvements via scaling don’t continue indefinetely, the pie will be split among any entity that can deliver value to another entity. The shape of “winners” will rhyme with many prior periods of history.

And by definition (at least the one I’m using), ASI’s abilities exceed the capacity of all human reasoning, so a confident prediction about it’s effects would be moot on arrival. I just pray it goes well.

The autonomous vehicle analogy for BI

The level names and their alignment with vehicle autonomy is sort of bastardized. Please allow for this, remembering that the original name of the “Level 0: no driving automation” vehicle is “automobile”³.

The Past

Level 0: Mental BI 🧠 - personal experience in bartering and such
- at this point, there’s no side quest of doing data analytics - just gut instincts and perhaps a relatively small amount of mental math
Level 1: Manual BI ✍️ - using one’s hand to manipulate pencil and paper
- the days when “computor” referred to a person and the tools were calculation tables, mechanical calculators, slide rules, etc.

The Present

Level 2: Computational BI 💻 - spreadsheets, Tableau, etc.
- a human does the high-level planning and synthesis, but the bulk computations are run by a computer, allowing for vastly more ambitous analytical projects
Level 3: Copilot for BI 🦾 - ChatGPT, Claude, etc.
- throughout the process, reasoning tasks are split between a person and software

The Future

Level 4: Autopilot for BI 🎮 - this is what we are working on now
- input natural language and get back useful analytics and insights - both quantitative and qualitative
- pithy - automates answering questions
Level 5: Full Self-Driving BI 🤖 - this is where we are going
- a proactive system which takes initiative to bring you insights (also answers questions, of course)
- pithy - automates both asking and answering questions

The Singularity

Level 6: Software as autonomous company 🌀 - we are not working on this
- What does it mean for the software to be fully autonomous? Do humans own it? Regulate it? Know about it?
- We don’t understand what it would be like to approach this moment, to be in this moment, or to go beyond this moment. It might happen, but it’s very confusing to reason about. Maybe Joscha Bach has something interesting to say about this?

Full Series: CHRT - Autopilot for Analytics (6 posts)
- Next Post: (2 of 6) CHRT.com - Repos, Demos

Footnotes

Before Jan 1, 2030↩︎
If you like zen or asymptotes - FSD BI will never be perfect, but it will always get better↩︎
The automobile moves itself. But doesn’t a horse move itself? From ~1920-2020, the movement was “auto” but the driving (observing the environment and controlling the steering and power actuators) was manual. Now some automobiles can do the driving themselves (auto-drive-o-mobiles?). But they don’t yet plan where to go or what to bring…what will those vehicles be called? Auto-reasoning-o-drive-o-mobiles?↩︎

--- title: "(1 of 6) CHRT.com - Intro" author: "Aaron Carver" description: "Autopilot for Analytics" date: "Nov 12 2024" date-format: "MMMM YYYY" # image: "img/image.jpg" # image-alt: a great image categories: [Projects, Full Stack, Code, CHRT] lightbox: true format: html: code-fold: true code-tools: source: true toggle: false caption: none --- ### Project Intro - I built CHRT with [Kyle Reagan](https://www.linkedin.com/in/kyle-reagan-surfs/) - We're colleagues from McKinsey and have been building software together for a couple years - At the end of August 2024, Kyle and I talked about how the future of analytics was going to be: - much bigger than the present - dominated by AI agents - So we built a product that does agentic analytics - some parts of it are described in this series of posts - But we aren't working on CHRT / AI Analytics anymore. We're doing something else now. ### Post-Mortem - ... ### Future of analytics - The two coolest things in analytics that I can think of right (Nov 30, 2024) now are: - [Julius AI](https://julius.ai/) - like CHRT, but better. *"Chat with your files and get expert-level insights in seconds."* - [MLE-bench](https://arxiv.org/abs/2410.07095) - a reasoning model was given a laptop and GPU for 24 hours and tasked with Kaggle ML competitions - it medalled on 17% of them. So yeah, that's the future - Before we built the product, I wrote a brief essay about the future of analytics. - I expect that the two resources above will very much be mainstream before 2030. --- ## Towards Full Self-Driving (FSD) Analytics ### The Thesis FSD Analytics will be a very useful thing. It will do analytics for you. You will describe what you want, and it will generate it. Actually - that's what Autopilot for Analytics will do. FSD Analytics will offer much more. FSD Analytics will *ask* important, interesting, and insightful questions. Then, it will do the analytics, evaluate them, and discuss the findings with you. Soon^[Before Jan 1, 2030], FSD Analytics will outperform the median highly-talented person and thus will obviously dominate the market. ### Does FSD Analytics mean no humans "in the loop"? Humans will definetely still be in the loop. To illustrate this point, here's a question with a false premise: Do FSD cars eliminate the need for humans in cars? Of course not! The false premise is assuming that the only reason a human would be in a car is to move the car - like in long-haul trucking. But many billions of miles of driving is done for the purpose of moving the human, so the FSD involved there has no change on the need for having a human in the vehicle. They just won't need to stare out the windshield and operate the power and steeting. Similarly, many billions of analytics charts and report are generated each year for the purpose of informing humans. They'll be able to get informed by data without needing to stare at the screen and control the keyboard. ### Is FSD BI the same as AGI? FSD BI is very different from AGI. I say that confidently, even without a consensus definition of AGI. Let's belabor the cars analogy: FSD vehicles are not AGI. They can drive to the store and back on their own, but they clearly can't do "anything a human could", or other similar definitions of AGI. FSD BI will be very useful, but is not AGI^[If you like zen or asymptotes - FSD BI will never be perfect, but it will always get better]. ### Will AGI or ASI kill the need for analytics? I think AGI will drastically increase the amount of analytics that is produced and consumed. I think it's a Jevon's Paradox situation - efficiency driving affordability driving more feasible use cases driving increased demand. There's certainly a big question about who will produce those analytics - the ~5 entities building trillion-dollar computer clusters to train huge models, or smaller and scrappier companies building product for their users. If the Scaling Laws (or at least some scaling law) continue indefinetely, I'd bet everything on the single most well-resourced frontier lab (or the top few) that operate(s) under regulations which do not limit matrix multiplication. If improvements via scaling don't continue indefinetely, the pie will be split among any entity that can deliver value to another entity. The shape of "winners" will rhyme with many prior periods of history. And by definition (at least the one I'm using), ASI's abilities exceed the capacity of all human reasoning, so a confident prediction about it's effects would be moot on arrival. I just pray it goes well. ### The autonomous vehicle analogy for BI The level names and their alignment with vehicle autonomy is sort of bastardized. Please allow for this, remembering that the original name of the "Level 0: no driving automation" vehicle is "**auto**mobile"^[The automobile moves itself. But doesn't a horse move itself? From ~1920-2020, the movement was "auto" but the driving (observing the environment and controlling the steering and power actuators) was manual. Now some automobiles can do the driving themselves (auto-drive-o-mobiles?). But they don't yet plan where to go or what to bring...what will those vehicles be called? Auto-reasoning-o-drive-o-mobiles?]. #### The Past - **Level 0: Mental BI 🧠** - personal experience in bartering and such - at this point, there's no side quest of doing data analytics - just gut instincts and perhaps a relatively small amount of mental math - **Level 1: Manual BI ✍️** - using one's hand to manipulate pencil and paper - the days when "computor" referred to a person and the tools were calculation tables, mechanical calculators, slide rules, etc. #### The Present - **Level 2: Computational BI 💻** - spreadsheets, Tableau, etc. - a human does the high-level planning and synthesis, but the bulk computations are run by a computer, allowing for vastly more ambitous analytical projects - **Level 3: Copilot for BI 🦾** - ChatGPT, Claude, etc. - throughout the process, reasoning tasks are split between a person and software #### The Future - **Level 4: Autopilot for BI 🎮** - *this is what we are working on now* - input natural language and get back useful analytics and insights - both quantitative and qualitative - pithy - automates answering questions - **Level 5: Full Self-Driving BI 🤖** - *this is where we are going* - a proactive system which takes initiative to bring you insights (also answers questions, of course) - pithy - automates both asking and answering questions #### The Singularity - **Level 6: Software as autonomous company 🌀** - *we are not working on this* - What does it mean for the software to be fully autonomous? Do humans own it? Regulate it? Know about it? - We don't understand what it would be like to approach this moment, to be in this moment, or to go beyond this moment. It might happen, but it's very confusing to reason about. Maybe Joscha Bach has something interesting to say about this? --- - Full Series: [CHRT - Autopilot for Analytics (6 posts)](/blog.html#category=CHRT) - Next Post: [(2 of 6) CHRT.com - Repos, Demos](/posts/2024/chrt/02_repos_demos.html)