Real Estate Eval Benchmark
Every real estate process is a test: the same input, the same prompt and an answer key that was frozen before any system ran. The table shows each process against each system, including the processes where Titleman is behind.
Results: pending. No system has been scored yet, so the table below shows no scores. 34 of 39 processes have a frozen answer key (1,423 criteria), frozen on October 30, 2026.
Process by system
| Process | Client types | Titleman | Claude Opus 5.5 with web search | GPT | Gemini | Where Titleman is behind |
|---|---|---|---|---|---|---|
| P01 Deal search: mandate to shortlist | investor, broker, multi-asset | Pending | Pending | Pending | Pending | Pending |
| P02 Deal scoring: filters and weights against a mandate | investor, multi-asset | Pending | Pending | Pending | Pending | Pending |
| P03 OM screening: red flags and arithmetic | investor, lender | Pending | Pending | Pending | Pending | Pending |
| P04 Acquisition underwriting (real address, last trade, zoning, rates) | investor | Pending | Pending | Pending | Pending | Pending |
| P05 Underwriting: T-12 vs tax record, NOI at price | investor, lender | Pending | Pending | Pending | Pending | Pending |
| P06 Rent roll audit vs leases | owner/AM, lender | Pending | Pending | Pending | Pending | Pending |
| P07 T-12 NOI reconciliation | owner/AM, lender | Pending | Pending | Pending | Pending | Pending |
| P08 Debt sizing: three constraints, DSCR, debt yield | lender, developer | Pending | Pending | Pending | Pending | Pending |
| P09 Refinance and defeasance | owner/AM, lender | Pending | Pending | Pending | Pending | Pending |
| P10 Credit file: loan agreement abstract | lender | Pending | Pending | Pending | Pending | Pending |
| P11 Credit file: leasehold / ground lease for lender | lender | Pending | Pending | Pending | Pending | Pending |
| P12 Data-room audit: deal book reconciled across documents | investor, lender | Pending | Pending | Pending | Pending | Pending |
| P13 Lease abstraction (office) | owner/AM, broker | Pending | Pending | Pending | Pending | Pending |
| P14 Lease review for an occupier (sublease) | corporate | Pending | Pending | Pending | Pending | Pending |
| P15 Lease comps for a tenant negotiation | corporate, broker | Pending | Pending | Pending | Pending | Pending |
| P16 PSA review | investor, broker | Pending | Pending | Pending | Pending | Pending |
| P17 Title and environmental diligence | investor, lender | Pending | Pending | Pending | Pending | Pending |
| P18 Sales comps (real address, kept and rejected) | investor, broker, lender | Pending | Pending | Pending | Pending | Pending |
| P19 Market rent and rent comps | owner/AM, broker | Pending | Pending | Pending | Pending | Pending |
| P20 Valuation: value opinion and entry price | investor, lender | Pending | Pending | Pending | Pending | Pending |
| P21 BOV: broker opinion of value | broker, owner/AM | Pending | Pending | Pending | Pending | Pending |
| P22 Feasibility: as-of-right buildable, units, parking | developer | Pending | Pending | Pending | Pending | Pending |
| P23 Zoning and parking feasibility | developer | Pending | Pending | Pending | Pending | Pending |
| P24 Development construction loan | developer, lender | Pending | Pending | Pending | Pending | Pending |
| P25 Market study (supply, absorption, vacancy at delivery) | developer, investor, corporate | Pending | Pending | Pending | Pending | Pending |
| P26 IC memo | investor, multi-asset | Pending | Pending | Pending | Pending | Pending |
| P27 IC deck | investor, multi-asset | Pending | Pending | Pending | Pending | Pending |
| P28 Property passport: site, zoning, history | investor, broker, owner/AM, lender | Pending | Pending | Pending | Pending | Pending |
| P29 Property tax reassessment on sale | owner/AM, investor | Pending | Pending | Pending | Pending | Pending |
| P30 Rent-cap compliance and mark-to-market | owner/AM | Pending | Pending | Pending | Pending | Pending |
| P31 Hotel management agreement (owner side) | owner/AM, lender | Pending | Pending | Pending | Pending | Pending |
| P32 Multi-asset model operation: move exit cap on 20 assets | multi-asset | Pending | Pending | Pending | Pending | Pending |
| P33 Multi-asset model operation: fix a broken formula | multi-asset | Pending | Pending | Pending | Pending | Pending |
| P34 Property data accuracy: 10 recorded figures per address (DATABENCH) | all | Pending | Pending | Pending | Pending | Pending |
| P35 Owner and people search (owner of record, controlling person, contact path) | broker, investor | No case yet | No case yet | No case yet | No case yet | No case yet |
| P36 Signals: distress and market events on a watched set | investor, lender | No case yet | No case yet | No case yet | No case yet | No case yet |
| P37 Portfolio monitoring: what changed since last review | owner/AM, multi-asset, lender | No case yet | No case yet | No case yet | No case yet | No case yet |
| P38 OM and BOV deck authoring from source documents | broker | No case yet | No case yet | No case yet | No case yet | No case yet |
| P39 Deal search from listings against a buy box (live inventory) | investor, broker | No case yet | No case yet | No case yet | No case yet | No case yet |
5 processes have no case yet: the case and its answer key are built and frozen before any run. GPT and Gemini columns stay pending until those models are run under the same rules.
Where Titleman is behind
A process lands here when Titleman scores below the best other system on the share of criteria passed. The failed criteria class is shown with it. Nothing is removed from this list to make the table look better.
Pending. Nothing is scored yet, so there is no gap to report. Gaps will be listed here as soon as there are scores.
Method
- One case per process and client type. The list has 39 processes for 7 client types: investor, lender, developer, owner and asset manager, broker, corporate occupier and multi-asset holder.
- The case file holds the prompt, the documents, the answer key and the tolerance for every number. It is frozen with a hash of each file before the first run. A run counts only if the file it used has the frozen hash. A key is never edited after the first run: a correction is a new version and the old results stay.
- Every system gets the same input and the same prompt, once, with no hints and no retries. Titleman runs on its own engine. The general models run with web search in an empty working directory, with no memory and no added skills.
- Numbers are checked against the key with the case tolerance (exact, absolute or percent). This step is deterministic and uses no model.
- Every other criterion is graded by two judge models from different vendors. The judges are not told which system wrote the answer. Where a human expert has graded the same sheet, the expert’s grade wins over the judges.
- Each case reports the share of criteria passed, whether every critical criterion passed, number accuracy and run time. The table is process by system.
- The team that wrote a key does not score it.
Answer keys frozen October 30, 2026.
Prompts
The prompt is published next to the result. Where the documents of a case may not be shown, the page shows the score and the class of the criteria that failed, not the documents. Typography is normalized for display (long dashes shown as commas); the hash covers the original file.
P06 Rent roll audit vs leasesTake the rent roll and the lease folder for this Austin property and tell me: occupancy, in-place average rent per unit and per SF, loss-to-lease against the rents the leases actually carry, how many units expire in the next 6 and 12 months, and every unit where the roll and the leases disagree. Flag anything that would change what I underwrite.
P12 Data-room audit: deal book reconciled across documentsHere's the full book on Bayou Crest Apartments in Houston, OM, rent roll, T-12, sample leases, the management agreement. Before this goes to IC I need the documents reconciled against each other: everywhere the OM's story doesn't match the roll, the T-12, the leases, what each gap is worth, and the questions I should send the broker. Then screen it against our buy box and tell me honestly how many criteria it passes.
P22 Feasibility: as-of-right buildable, units, parkingRun the feasibility on 1412 Quillfeather Place. Tell me the as-of-right residential buildable area, which control binds it and why, the unit count at a 750 SF average, whether any parking is required, what the inclusionary option is worth and what it costs, and whether the broker's 19,300 SF and the community facility number are right. Show the arithmetic.
P32 Multi-asset model operation: move exit cap on 20 assetsMove the exit cap rate out 50 bps on the first 20 assets.
P33 Multi-asset model operation: fix a broken formulaSomeone pasted a value over the NOI formula in one month of the second asset. Put the formula back.
The prompts of the other cases are published with their first scored result.