[Training a 200B LLM from Scratch] 3/25: What Is the Full Budget?

The purchase list for 256 servers already exceeds $100 million. What matters next is when those machines will have a home, who can make them work, and whether the money will arrive before each bill is due.

The first two articles defined the job: use 2,048 B200 GPUs to train a dense 200B model on 4T cumulative training tokens. At the training throughput selected in Part Two, the main run takes about 30.1 days, so we allow 31. This article puts servers, networking, the site, the team, and funding into one plan. Public prices and availability are current as of October 18, 2026.

Start with the equipment list: choose the models, count the units, and place them in a facility that can support them. Servers and switches have public prices. The next facility step is a written offer covering 4.5 MW, a handover date, and the price. Aligning equipment and site delivery gives us a basis for releasing the large orders.

First, decide when ownership is worth buying

For a single run, first ask a cloud provider for a whole-cluster quotation covering 31 continuous days. Lambda lists B200 clusters at $8.87 per GPU-hour for the 256-plus tier. Multiplying by 2,048 GPUs and 744 hours gives about $13.52 million. The RFQ should specify simultaneous access to all 2,048 GPUs, 31 continuous days, the required network, and delivery dates.

This company chooses ownership because it plans repeated research runs. The reference purchase uses Exxact’s eight-GPU B200 AMD configuration at $393,167.50 per server. There are 256 main servers and eight spares, for 264 machines costing $103,796,220. An alternative Intel configuration costs $423,857.50, or $111,898,380 for 264. Their CPUs, system memory, and other details differ, so they need the same acceptance workload. The central budget uses the first configuration, keeping the calculation tied to one specific machine.

Eight spare servers are the project’s chosen 3.125% repair allowance. The main run uses 2,048 GPUs, with another 64 replacing failed positions. GPUs are included in the server price. At ordering, put the configuration, quantity, delivery dates, warranty, and acceptance criteria for all 264 machines into the supplier-confirmed purchase list.

Cluster: A group of servers working together.

GPU-hour: The unit for renting one GPU for one hour. A whole-cluster quotation must also state the network and services included.

RFQ: A request for quotation, listing the requirements we send to a supplier for pricing.

Which switches should we buy, and how do they connect?

Network adapter (NIC): A component that gives a server a network connection.

Switch: A device that connects other devices so they can exchange data.

Port: One network connection point on a device.

InfiniBand: The high-speed networking system selected here for training machines to exchange data.

400G: A nominal transmission rate of 400 billion bits per second.

Bandwidth: The amount of data a connection can carry per unit of time.

Topology diagram: A diagram showing which devices connect to which.

The training network must continuously exchange results among 2,048 GPU network endpoints. This procurement reference uses 400G InfiniBand. The eight GPUs within a server have their own high-speed interconnect; across servers, each GPU maps to one 400G network adapter. Exxact’s system acceptance must cover the slots, connections, and per-adapter bandwidth for the eight specified NICs.

The switch reference is NVIDIA Quantum-2 MQM9700-NS2F. Colfax lists it at $34,555. It provides 64 logical 400G ports through 32 twin-port OSFP cages. Those 64 ports fit the topology below, and we can check the adapters, optics, and switches against the same 400G InfiniBand specification.

Divide the 256 main servers into eight groups of 32. In each group, the first network endpoint from every server connects to one leaf switch, the second to another, and so on through the eighth. Leaves are the switches nearest the servers. Every leaf then connects to all 32 shared spine switches, which connect the groups. Follow the gold path in the diagram to see a result travel from one group to another.

Each leaf has 32 ports down and 32 up, giving 12.8 Tb/s in each direction. Eight groups with eight leaves each give 64 leaves. Every spine connects to all 64 leaves, using its 64 ports. The total is 64 plus 32, or 96 switches. There are 2,048 server-to-leaf links and 2,048 leaf-to-spine links, for 4,096 400G links. Use this topology to test cabling, failure recovery, and training performance at acceptance.

At these public prices, the training fabric totals $16,841,184. Optics and fiber alone account for $10,146,816, making them worth pricing individually. The 3,072 twin-port switch modules provide 6,144 400G ports: 2,048 face the servers, and 4,096 terminate both ends of the leaf-to-spine links. Adding 2,048 single-port host modules equips both ends of all 4,096 links. Spare servers reuse the optic and cable at a failed position; request a quotation for additional optical spares with the supplier’s proposal.

Service scope changes the price. NADDOD’s same-model switch bundle with three years of service costs $48,592 per switch, adding $1,347,552 across 96 switches compared with the catalogue reference above. Compare offers with the same service scope. On October 18, 2026, the NIC page showed 120 units in stock. We need 2,112, so the next supplier request is a batch-by-batch schedule covering the full quantity.

Two site design tasks complete the plan. First, 30 m fiber is a specific pricing example. Longer routes need different lengths; routes beyond the reach of this multimode optical design need a matched single-mode design. Second, the 96 switches use all data ports, so network management software, host resources, and high availability need a separate design. Storage and management networking retain a $1–2 million allowance, and shared storage a $2–4 million allowance. The next RFQ should complete their model selection, service scope, and pricing.

Leaf: A lower-tier switch next to the servers.

Spine: An upper-tier switch that connects the groups of leaf switches.

Tb/s: A trillion bits per second, a unit of network bandwidth.

Optical module: A component that converts electrical signals into light so fiber cables can carry data.

OSFP: The socket and module form factor used for the optical modules in this plan.

Which actual facilities should we ask?

Rack: A frame that holds servers.

kW (kilowatt): A unit of power at a given moment. One kW equals 1,000 watts.

MW (megawatt): A larger unit of power. One MW equals 1,000 kW.

kWh (kilowatt-hour): A unit of energy consumed over time. The facility bills this energy according to the contract.

Using the equipment webpage’s 14.25 kW power estimate per server, 264 servers imply about 3.76 MW. Adding networking, storage, and headroom, we ask for 4.5 MW of IT capacity. That is power delivered to equipment, before cooling and other facility use. The operator must also approve server counts, power per rack, airflow, floor loading, and cable routes.

The first RFQ target is Flexential Atlanta-Douglasville 2 in Georgia. Its published hall size is exactly 4.5 MW, with density above 80 kW per cabinet, making it a close technical match. The key RFQ requirement is an allocated 4.5 MW hall. An August 2026 financing announcement still describes a 36 MW facility in that market as under construction. We need confirmation of the particular phase that could be delivered.

These are the selected RFQ candidates. The project is at proposal preparation. Put the deliverable 4.5 MW allocation, rack plan, handover, rent start, minimum term, deposit, energy billing, onsite service, and remedies into one RFQ. Responses can then be compared on who can deliver the complete package when needed.

We can also narrow the price benchmark to the right market size. CBRE’s H1 2026 Atlanta report, in the 3–10 MW tier, gives market asking rents of $150–175 per kW-month. Using the midpoint of $162.50 produces 4,500 × $162.50 × 9 months = $6,581,250. Flexential’s project quotation needs to specify high-density fit-out, special cooling, and additional services alongside this market benchmark. A $250 actual rent would add $3,543,750 over nine months versus the central case.

The calculation assumes rent plus separately billed electricity. CBRE’s market report also distinguishes this structure from leases that pass more costs to the tenant. If a supplier includes electricity, combine those rows. Add minimum power payments, passed-through taxes, or deposits when the terms require them. Compare total prices using that complete billing scope.

Price electricity at $0.10 per kWh and an annual average of 730 hours per month. Budget full-cluster IT operating load at 3.3MW and use the facility’s target PUE of 1.4. A full operating month costs 3,300 × 1.4 × 730 × $0.10 = $337,260. Months 4 and 5 contain only the first batch, so allocate half the operating load; Months 6–12 use the full load. First-year electricity totals $2.6981 million. Metered installation and training loads then update the remaining budget.

PUE: Total facility energy divided by IT equipment energy. At a PUE of 1.4, every 1 kWh used by equipment requires 1.4 kWh for the facility including cooling.

Reserved capacity: The power capacity held for the project, which is 4.5 MW here.

Operating load: The power the running equipment needs. The budget uses 3.3 MW when the full cluster is operating.

Which work fits colocation, self-build, and multiple sites?

A self-build needs a separate schedule. JLL’s 2026 outlook reports average grid-connection waits above four years in major markets. Its approximately $11.3 million per MW construction figure is based on shell and core for a 50 MW air-cooled facility, excluding land and IT equipment. Our 4.5 MW self-build needs its own estimate, including land, design, grid connection, construction, commissioning, financing, facility staff, and residual asset value over several years.

The first-year project prioritizes colocation with established power and a written handover date. Secure that date, then schedule procurement, hiring additions, and rent start. A site delay triggers a coordinated reschedule while small experiments and data preparation continue. Manage existing orders and leases under their contracts.

Multiple data centers can handle separate data preparation, independent experiments, synthetic-data generation, and evaluation. Splitting the same synchronized main run across cities requires a new dedicated-link and parallelism design, followed by throughput measurements. Use that measured throughput to recalculate the previous 31-day training schedule. The procurement reference keeps the main run in one campus; even buildings within one campus need distance and dedicated-link validation. The inter-building links must meet the training fabric’s bandwidth and latency requirements.

Colocation: Putting our servers in a facility operated by another company and renting space, power, and cooling.

Latency: The time for data to reach its destination. It determines how long a machine waits for a result.

Price the twenty positions individually: who joins first?

Distributed training: Several machines cooperating on one training run.

Evaluation: Using test tasks to measure model performance.

Post-training: Further training of a base model to improve task performance and instruction following.

SRE: Site reliability engineering, which keeps systems running reliably through monitoring, incident response, and recovery.

M1: Project Month 1. M2, M3, and later labels indicate subsequent project months.

Twenty is the initial engineering team: five in model work, four in data, three in distributed training, three in infrastructure, four in evaluation and post-training, plus a programme lead. The plan reuses existing training software and assigns facility operations and physical installation to external partners. More work, new algorithms, or weaker external support would require a different headcount.

Set pay role by role using public US recruiting bands. Training engineering roles list base pay of $200,000–290,000, GPU systems roles $200,000–300,000, and data engineering roles $182,000–242,000. Use these bands to build the budget, then set offers against each candidate’s responsibilities and experience. A frontier-lab chief scientist would receive a separately budgeted package. This package targets people with large-scale training or cluster experience who want technical ownership in an early team. The 20% lead bonus and 15% engineer bonus are our choices for responsibilities and cash planning. Candidates will also consider research scope, compute access, and the team, so discuss equity and technical decision-making authority together.

The monthly plan starts after the first two leads have accepted their roles and initial funding is available. Pay the programme and model leads in Month 1; target Month 2 for the other four workstream owners, with searches starting earlier. The large purchase requires infrastructure and distributed-training review, so the decision moves if those owners are missing. SHRM’s 2026 recruiting benchmark reports a 39-day median for nonexecutive time-to-fill, a cross-industry measure ending at offer acceptance. The joining plan also needs onboarding and time to become productive.

A competitive offer brings four parts together: base salary, performance bonus, sign-on payment, and equity. The six leads receive a target bonus of 20% of base; the other fourteen positions receive 15%. Pay the award in Month 12, prorated by months worked that year. Sign-on payments are $50,000 for leads and $25,000 for other positions, paid when they join. Keep the 25% employer payroll-cost and benefits allowance on base pay, then budget an additional 8% employer allowance on bonuses and sign-on payments. Benefits are counted once, and each additional cash payment has a place.

Headcount rises through 2, 6, 8, 10, 14, and 17, reaching 20 in Month 7. Year 1 includes $3.5703 million of base pay, $0.8926 million of existing employer costs and benefits, $0.6086 million of performance bonuses, $0.6500 million of sign-on payments, and $0.1007 million of incremental employer allowances. Total personnel cash is $5.8221 million, an increase of $1.3593 million over base pay and benefits alone. For example, an engineer joining in Month 7 on $245,000 base earns 245,000 × 15% × 6 ÷ 12 = $18,375 in first-year bonus. This lifts Month 12 team cash payments to $1.1571 million.

Reserve a 10% employee equity pool on a post-financing, fully diluted basis. Initial awards in the table total 4.35%, leaving 5.65% for later hires and additional awards. Vest over four years, with 25% at the first anniversary and monthly vesting thereafter. Equity attracts people through participation in future value. Its current cash cost is zero, while shareholders bear dilution. The first two leads are also employee positions in this schedule; founder ownership is determined separately. Carta’s compensation report published in 2026 found that from January 2024 to February 2026, median initial equity grants for AI/ML engineers grew 31%, compared with 9.1% growth in median salaries. That is another reason to build cash and equity into the recruiting package together.

The programme lead approves procurement and payment; technical owners define specifications and acceptance. External legal, procurement, and finance specialists handle contracts, delivery responsibilities, tax records, and payroll services. Allow $300,000–600,000 annually pending engagement quotations. The facility operates the building, power, and cooling; the integrator performs the contracted installation, cabling, and acceptance; the equipment supplier provides the contracted warranty. Three internal infrastructure engineers own configuration, monitoring, diagnosis, and recovery. Put round-the-clock onsite replacement and response into the external contract; the onsite service team provides the shift coverage.

Equity pool: Ownership set aside for employee incentives.

Fully diluted ownership: A share calculation that counts existing shares and rights such as options that could become shares.

Vesting: Earning the agreed rights over time.

Dilution: A reduction in existing owners’ percentage of the company when more shares are issued.

After funding, when can training begin?

The procurement plan brings 264 servers onsite in two batches of 132, each with 128 training servers and four spares. Target procurement review in Month 2, deliveries in Months 4 and 6, full-scale acceptance in Month 7, and the 31-day main run in Month 8. This is the target sequence to coordinate with suppliers and the facility. Two deliveries support staged installation; payment milestones belong in the purchase contracts. Install all 32 upper-tier switches and shared storage with the first batch, then connect the second batch to that system.

Before ordering, obtain supplier delivery proposals against these target months and align them with site access. A later site handover moves equipment delivery, installation, rent start, and training together. Same-month equipment and site delivery reduces storage, insurance, and capital occupancy. Add those costs when earlier equipment arrival is needed. Rent the full hall from Month 4 to secure space for the complete 4.5MW allocation. Budget half-cluster operating power in Months 4 and 5 and full-cluster power from Month 6, with actual bills following the meter.

The data lane can start with real materials. FineWeb provides public samples and streaming access; its current card lists over 18.5T English tokens using its GPT-2 tokenizer basis. FineWeb2 provides multilingual material and reports over three trillion words, to be counted as training tokens using the chosen tokenizer. These are public datasets originating in 2024 and 2025, with current cards available, and they support immediate sample testing. The project must still review sources and usage conditions, recount with its own tokenizer, and check cross-source overlap and evaluation contamination.

In Months 1–2, measure download, cleaning, and tokenization throughput, audit retained samples, and test the mixture with a small model. Use measured sample throughput to schedule bulk preparation in Months 3–6 and track progress against that target. At Month 7 data acceptance, require a source manifest, frozen tokenizer and mixture, accepted token inventory, repeat-sampling policy, and sustained data-feeding test. The 4T target is cumulative training volume. List the accepted unique-token inventory together with repeat-sampling counts to show how the 4T total is formed.

The actual start is the latest of equipment acceptance, site acceptance, data acceptance, and people-and-recipe readiness. Target Month 7 for full-cluster tests of sustained data supply, communication, recovery, and training speed. Start the main run when all four lanes are ready. When delivery dates change, reschedule installation and training, then update payments under each contract. Keep deposits, rent, and service payments already due in the cash schedule.

Tokenizer: A tool that decides how text is split into tokens. Different tokenizers produce different counts.

Deduplication: Identifying and removing repeated training content.

Evaluation contamination: Test questions or answers entering training material and inflating measured performance.

Bring equipment and operations into one budget

Beyond the priced equipment, allocate budgets and request quotations for these items: $0.5–1 million for small cloud experiments in the first three months; $1–3 million for annual data licensing and external processing; and $1–2 million for software, network-management licenses, external cluster services, and operating insurance. Server warranty is included in the system price. A switch package with three-year service reduces the corresponding service allowance. Freight and transit insurance receive $0.5–1 million, installation and acceptance $1–2 million, and hardware tax a provisional 8% allowance. Replace the tax amount after location, exemptions, imports, and contract structure are established.

Hardware totals $125.14 million. Taxes, freight, and installation bring one-time procurement funding needs to $137.40 million. First-year operations cost $19.80 million, including $5.82 million of personnel cash with performance and sign-on bonuses. Start by covering this procurement and operating plan with equity funding; add equipment debt after obtaining customer-specific credit approval.

First-year procurement and operations total $157.20 million. Add a 10% risk allowance of $15.72 million, giving a funding coverage target of approximately $172.92 million. This answers how much funding the plan needs to raise. Schedule actual bank receipts against the investment agreement and each equipment and facility contract.

One-time procurement: The spending required to buy and install equipment.

Operating payments: Recurring costs such as pay, rent, and electricity.

Equity funding: Money investors put into the company in exchange for ownership.

Equipment loan: Debt used to buy equipment, repaid with principal, interest, and fees under the contract.

Can a new company obtain equipment financing?

Start with how the financing provider gets repaid. A compute provider can point to customer contracts and recurring service revenue as the source of monthly payments. This company spends first to train its own model, with revenue coming later. A lender must assess shareholder capital, available cash, the credibility of future revenue, and recovery from selling the equipment. GPUs have resale value; the company still needs a credible source of repayment.

Data Sales advertises equipment lease lines of up to $50 million. Its Ori case study shows how such financing happens: Ori had spent five years developing its platform, acquired GPUs to fulfill customer contracts, began with a lease in the tens of millions of dollars, and expanded the relationship as the business grew. Customer demand and an operating foundation matter. For a new company without comparable revenue and customer contracts, submit financial information and a business plan to obtain a company-specific credit decision. The advertised maximum helps select providers to approach.

There are providers to approach as a startup. NFS Capital’s official guidance explicitly considers startups and pre-revenue businesses through individual credit decisions. Depending on credit, asset loans may involve additional support such as cash accounts or securities, or structures involving warrants. Exxact’s sales FAQ also offers partner leasing and customized financing, with terms depending on location and credit history. Obtain the approved amount, down payment, funding price, guarantees, and draw conditions together to establish a complete proposal. As of October 18, 2026, this project has obtained no customer-specific quote or credit approval; the primary funding table includes zero equipment debt.

After receiving a quote, check guarantees, funding-release conditions, early repayment, and the end-of-term buyout. Then enter each principal, interest, and fee payment into the cash schedule. Use the quoted rate and include fees in the total cost of funding. That establishes how much first-year cash financing preserves and how much payment it moves into later years.

Credit approval: The amount and conditions a lender agrees to offer after review.

Drawdown: Actually taking money from an approved borrowing arrangement.

Principal: The amount borrowed.

Interest rate: The rate used to calculate interest. Financing fees are counted separately.

Guarantee: A guarantor’s agreed responsibility to support repayment.

Collateral: Assets pledged to support repayment.

Warrant: A right to buy company shares on agreed terms.

Separate the payments through the contracts

With equipment arriving in two batches, purchase contracts can allocate payments to an advance, shipment or delivery, and acceptance. Suppliers establish the amount and date at each milestone, and the funding plan follows those obligations. Colfax’s public terms accept cash or approved credit, with separately accepted purchase terms able to change the payment arrangement. NADDOD’s Net 15, 30, and 45 terms also require credit approval. These short payment terms and three-year equipment finance address different periods of cash use.

Propose two deliveries with milestone payments, and secure funding commitments sufficient to cover payment obligations before placing a non-cancellable order. If the supplier requires an advance, move the cash receipt earlier. If it agrees to an acceptance holdback, schedule that balance under the acceptance terms. Show the full $137.40 million procurement and installation requirement now; allocate it to months through the contract quotations. This preserves room to negotiate and makes the obligations created by each order visible.

First plot operating payments that follow the project schedule. Hiring, cloud experiments, and data services start in stages; target rent from Month 4, full-cluster operating load from Month 6, and performance bonuses in Month 12. The chart separates procurement funding from monthly operations: contract terms will allocate the upper section, while the lower section follows the current project schedule.

The monthly funding formula remains simple: contract payments due + current operating payments + the next two months of operating buffer − available bank cash. Call committed funds when this leaves a gap. Contracts supply the equipment payment dates and amounts; the table supplies operating payments. Month 12 bonuses enter the cash buffer in advance. The buffer is part of the $15.72 million risk allowance, avoiding double-counted funding.

Staged funding needs support in the investment agreement. Align committed amounts, funding notice periods, conditions for later contributions, and payment obligations when signing. If later contributions still depend on a new investment decision, size procurement to the funding already secured. Separate payments where the contracts permit it, and give every non-cancellable purchasing obligation a funding source that can cover it.

Cash flow: When money enters and leaves the bank account.

Funding commitment: An investor’s obligation to provide money under an agreement.

Available cash: Money that has already arrived in the account and can be used.

Payment term: The period from the contract’s specified starting date to the payment deadline. Net 15, Net 30, and Net 45 mean payment within 15, 30, and 45 days respectively.

The decisions we can make now

Keep 264 B200 servers and a 20-person engineering team. First-year procurement and operations total approximately $157.20 million, and the funding coverage target including the risk allowance is $172.92 million. Equity funds the primary plan, with zero equipment debt and zero year-end equipment debt balance. First-year personnel cash is $5.82 million, including base pay, performance awards, sign-on payments, and employer costs; equity awards follow the separate hiring plan.

Next, obtain delivery and payment quotations for both equipment batches and handover and rent terms for the 4.5MW facility. Then place each part of the $137.40 million procurement requirement into its payment month. Deliveries in Months 4 and 6, acceptance in Month 7, and training in Month 8 are targets for coordinating the parties. Keep equipment financing as an additional quotation route; replace the funding plan with its actual down payment, rate, and repayment schedule after obtaining company-specific approval and complete terms.