Most of what is written about this is about products, and products date badly. This page is about the three things underneath them that have not dated: a published architecture, a set of open frameworks, and the silicon that made scale an engineering problem.
People type "Google AI revolution" after reading a headline, and headlines rarely say what changed. The useful answer has three parts, and they arrived in that order: a published architecture, a set of open frameworks, and custom silicon to run them on. Consumer products came fourth, and the products are the part that goes stale fastest.
The architecture is the transformer, described in a 2017 paper by a team of Google researchers. The frameworks are TensorFlow, open sourced in 2015, and JAX, which followed it as the substrate for a great deal of research work. The silicon is the Tensor Processing Unit, disclosed publicly in 2016, a chip designed around the single operation that dominates neural network training. Running alongside all of that, DeepMind, which Google acquired in 2014 and later merged with Google Brain, produced a research line that includes AlphaGo and AlphaFold.
Compressed into one sentence: Google published the design that most modern language models use, gave away much of the software they are trained with, and built hardware that turned training at scale into an engineering problem rather than a procurement one.
That is deliberately a statement about capability rather than about any particular assistant. Assistant products get renamed, repositioned and replaced, and a page that pinned itself to one of them would be wrong within a year. The architecture, the framework ecosystem and the accelerator economics are what a working engineer still has to understand several years from here.
Before it, the standard way to model a sentence was recurrent. A network read one token, updated a hidden state, then read the next. That design has two problems. The hidden state is a bottleneck, so dependencies between distant words degrade. And the reading is inherently sequential, so the work cannot be spread across a sequence no matter how much hardware you point at it.
Attention already existed as a patch on recurrent translation models: let the decoder look back across all encoder positions and weight them by relevance. The contribution of the 2017 paper was to delete the recurrence and keep only the attention. Each position produces a query, a key and a value, every position attends to every other in one large matrix operation, and position information has to be added explicitly because the network no longer has an order in which it reads.
Two consequences followed, and between them they are the whole story. The path between any two tokens becomes a constant length instead of growing with the distance, so long range dependencies survive. And the computation collapses into a stack of large matrix multiplications, which is precisely what a GPU or a TPU is built to execute. Training stopped being limited by sequence order and started being limited by how many accelerators you could keep busy.
Google researchers published BERT in 2018, which pretrained a bidirectional transformer encoder by masking tokens and predicting them, and showed that one pretrained model could be fine tuned for many different language tasks. Pretrain, then adapt, is the working pattern almost every practitioner uses today, and it dates from this period.
The honest footnote is that the architecture is public. Anybody can read the paper and implement it. That is exactly why a design published by a Google team underpins systems built by laboratories competing directly with Google, and it is a good reminder that in this field the published idea travels much faster than the product.
TensorFlow was open sourced in 2015 and did something quite specific: it put a production grade training and serving stack into public hands. You describe a graph of tensor operations, gradients come from automatic differentiation, the same graph runs on a CPU, a GPU or a TPU, and there is a serving path for putting the result behind an API. Keras became the high level interface most people actually touch.
JAX takes a different route to the same place. It offers a NumPy style array API plus a small set of composable function transformations: take a gradient, compile a function for the accelerator, vectorise a function over a batch, and shard a computation across devices. Because the style is functional and free of hidden state, those transformations compose cleanly, which is why research code that has to be both readable and fast tends to end up here.
Both feed the same compiler, which fuses operations and emits code for the target accelerator. That is the layer where the abstraction leaks in a useful way. You write array mathematics, and the compiler decides what the kernels look like. Knowing that this layer exists is what separates an engineer who can make a slow training run fast from one who can only wait.
| Where the fingerprints are | What it is | Why it mattered |
|---|---|---|
| Word embeddings, 2013 | An efficient way to train dense vectors for words | Established that meaning could be a vector, and that similarity was a geometric question |
| Sequence to sequence, 2014 | Encoder and decoder networks for translation | Reframed translation as a single learned model rather than a pipeline of components |
| TensorFlow, open sourced 2015 | A tensor and deep learning framework with a serving path | Made a full training and deployment stack available outside a research laboratory |
| TPU, disclosed 2016 | An accelerator built around a matrix multiply unit | Changed how quickly experiments could be run, and therefore how many could be run |
| Transformer, 2017 | Self attention with the recurrence removed | Made sequence training parallel, which is what made scaling practical |
| BERT, 2018 | A bidirectional encoder pretrained by masking tokens | Made pretrain and fine tune the default recipe for language understanding |
| JAX | Composable transformations over NumPy style code | Became a common substrate for large scale research experiments |
| AlphaFold | Protein structure prediction from an amino acid sequence | Turned a decades old problem in biology into a usable everyday tool |
The practical lesson for a learner is that the framework is not the skill. Tensors, automatic differentiation, a loss, an optimiser, batching and a training loop are the skill, and they look almost identical in PyTorch. An interviewer asks whether you can diagnose a loss that refuses to come down. Nobody asks which import statement you used.
The forward and backward pass of a neural network is dominated by matrix multiplication. A general purpose processor spends most of its transistor budget and most of its energy on machinery that does not help with that at all: branch prediction, deep cache hierarchies, out of order execution. A TPU strips those back and builds around a matrix multiply unit arranged as a systolic array, where operands flow through a grid of multiply and accumulate cells so that each value is reused in place instead of being fetched from memory again and again. At this scale it is memory movement, not arithmetic, that burns the power.
Two further design choices matter as much. The first is reduced precision. Google introduced bfloat16, a 16 bit floating point format that keeps the exponent range of the 32 bit format and gives up mantissa bits instead, because training tolerates coarse precision far better than it tolerates numbers falling out of range. The second is interconnect. Chips are wired directly to one another so that a large group behaves as one machine, which is what allows a model to be split across many devices without the network between them becoming the bottleneck.
The economic consequence is the part worth internalising. When one organisation owns the chip, the data centre and the compiler, the marginal outlay for an additional experiment is electricity and engineer time rather than a purchase order placed with somebody else. In a field where progress comes from running an enormous number of experiments and keeping the few that work, that difference compounds. It is also the plainest explanation for why the frontier of this work sits with a small number of organisations, and that is a fact about capital and infrastructure rather than about where the talent is.
Notice what that argument does not say. It does not say you need this hardware to be employable, and it does not say the interesting work is only at the frontier. It says the opposite: the expensive layer has been built and rented out, so the value that remains for everybody else is in the layer above it.
DeepMind was a London research company that Google acquired in 2014, and it was later merged with Google Brain into a single organisation. Its line of work is quite separate from the language model line, and it is the stronger argument that what happened here is a scientific shift rather than a product cycle.
AlphaGo beat a leading professional Go player in a five game match in 2016. The interesting part was never the scoreline but the method: a policy network to propose plausible moves, a value network to judge a position, and tree search to look ahead, trained first on human games and then by playing against itself. Successor systems dropped the human games entirely, learned from self play alone, and then generalised to chess and shogi with the same recipe. The lesson that carried forward is that search combined with a learned evaluation function is extremely strong wherever the rules are known exactly.
AlphaFold aimed the same instincts at a problem with no clean rules. A protein is a chain of amino acids that folds into a three dimensional shape, and that shape largely determines what the protein does. Predicting the shape from the sequence had been an open problem in biology for decades. AlphaFold combined evolutionary information drawn from related sequences with attention over the residues, and at the community assessment held in 2020 it predicted structures at an accuracy competitive with experimental determination for a large share of targets.
It also returns a confidence estimate for each residue, which matters more than it sounds. A prediction you can trust selectively, region by region, is usable in a laboratory in a way that a single unqualified answer is not. The predicted structures were then published as an open database in partnership with the European Bioinformatics Institute, and that is the step that changed day to day work for biologists who will never train a model themselves. In 2024 the Nobel Prize in Chemistry recognised computational work on proteins, with one half of the award shared by two DeepMind researchers for protein structure prediction.
For most of its history a search engine did one job in three stages: understand the query, retrieve candidate documents, rank them. Language models changed all three, and this is the part of the story that touches everybody, not only engineers.
Understanding came first. Google moved Translate onto a neural system in 2016, and from 2019 it applied transformer based language understanding to ranking. That is the point at which matching stopped being mostly about which words were present and started being about what the sentence meant. Small function words began to change results, because a preposition can invert the intent of a query without changing a single content word.
Retrieval changed next. Embeddings let a document be found because it is about the same thing as the query, even when the two share very few words. That is the same vector idea from word embeddings, applied at the scale of whole documents, and it is also the mechanism behind the retrieval step in almost every AI application built today.
Answering changed last and most visibly. A results page began to include a synthesised answer composed from retrieved sources rather than only a list of links. That is retrieval augmented generation in a consumer product, and it rewrites the contract between a search engine and a website: the engine reads the page and returns a summary, and the reader may never arrive. For anyone who publishes, being visible increasingly means being the source an answer is built from, not merely holding a position in a list.
Multimodality is the other shift. A model that accepts images, audio and video alongside text does it by encoding every modality into the same embedding space, so a single transformer stack can attend across all of them at once. That is what allows a phone camera to become a query, and a screenshot or a recording to be a legitimate input to a question. The product names attached to this have changed more than once and will change again, which is the strongest possible argument for learning the mechanism rather than the interface.
Here is the honest version. Training a frontier model requires an accelerator fleet, a data centre and a research organisation, and that is unlikely to change. It is not the job market for almost anybody reading this, and aiming at it is not useful advice.
What did open up is the layer above. Every organisation that wants to use these models has to do the same unglamorous work: get its own data in order, choose a model, wire retrieval to its own documents, evaluate whether the output is actually right, handle the cases where it is not, and put the thing into production without leaking anything. That work is being done in Technopark in Thiruvananthapuram, in Infopark in Kochi and in Cyberpark in Kozhikode, and by Kerala teams delivering for clients abroad. It needs engineers, not researchers.
Our Data and AI track covers that ground in two shapes. Data Science and AI is the longer route through statistics, machine learning and deep learning, and it suits you if you want to build and train models yourself. AI and Generative AI is the applied route for developers and working professionals who need to build on top of models that already exist: prompting, retrieval, fine tuning, evaluation and deployment.
One last piece of advice about pace. It is tempting to chase every announcement, and it is a waste of a year. The half life of a product name in this field is very short. The transformer has been the default design for most of a decade, and the hiring conversations we hear about are still about fundamentals plus one project you can defend line by line.
The applied route. Prompting, retrieval, fine tuning, evaluation and deployment for developers and working professionals building on existing models.
View course → Data Science and AIThe longer route through statistics, machine learning and deep learning, for people who want to build and train models rather than only consume them.
View course → All Data and AI coursesThe full track, from analytics foundations through to applied AI, so you can see where each programme starts and what it assumes you already know.
Browse the track →