Preparing Structured Data for Ingestion

<aside> 🧭

Nav Links

</aside>

Updated Nov. 17, 2025

This article uses a Fictional Private Markets Intelligence Platform as an Example

Organizations with structured or semi-structured datasets—such as company directories, investment databases, deal trackers, or fund analytics products—should prepare their content for Cashmere by converting each entity into a single materialized article view and submitting it to the POST /omnipub endpoint with correct metadata.

This guide uses a fictional Private Markets Intelligence Platform as the running example. In this fictional platform, the core entities include:

Companies
Investors
Funds
Service Providers
Deals

These entities live in their own relational tables but are presented to users as rich, joined profile pages. Cashmere needs those unified profile pages—converted to HTML or Markdown—as the ingestable content units.

1. Materialize Each Entity Into a Single Article View

A. How relational data becomes a profile page

In a relational system, an entity exists as a single row in a table, but the profile page users see is constructed from many joins.

Example: A Company profile may include:

Company details (from the companies table)
Deals involving the company (from deals)
Investors in those deals (from investors)
Funds associated with those investors (from funds)
Service providers participating in related deals (from service_providers)
Press releases or relevant news (from news)

Similarly, an Investor profile includes:

Investor details
All deals the investor participated in
All companies tied to those deals
Any funds the investor manages

These views naturally contain cross-linked content and repeated rows.

This repetition is correct and required.

B. Repeat data everywhere it appears

When you materialize the profile into an article view:

Do not de-duplicate entities
Do not reference other entities indirectly
Do not assume “the LLM can fetch that elsewhere”

Example:

If an investor made three separate investments into a company across three rounds, the Company profile should list:
- That investor three separate times, each with the correct round and amount
On the Investor profile, the same company may appear multiple times if they participated in multiple deals
On the Deal profile, show all participants, even if they appear in other profiles

This gives Cashmere a complete, self-contained, text-based representation of the relationships that exist in your dataset.

C. Remove internal/self-referential links

Your application UI likely has links to other profiles (e.g., clicking a Company name takes you to the Company profile).

For Cashmere ingestion:

Remove all internal hyperlinks
Replace them with plain text
Only keep URLs that point to external sources (e.g., news articles, filings)

Internal links confuse LLM interpretation of the content and undermine the goal of directing users to your canonical source_url.

D. Output the profile as HTML or Markdown with clear sections

Use headings to separate logical segments:

title: Apex Robotics
external_id: company_145726
source_url: <https://exampledata.com/companies/apex-robotics>
creation_date: 01/15/2025

# Apex Robotics
Apex Robotics is a Seattle, WA-based company founded in 2014. Apex Robotics develops autonomous robotic systems for industrial logistics. The company operates in the Information Technology sector within the Robotics group.

Apex Robotics is currently Generating Revenue and is Venture Capital-Backed. Apex Robotics employs 320 people and reported $48M in revenue for FY2024.

## Apex Robotics Deal Activity
### Apex Robotics Series A — Jan 2020
In 01/15/2020, Apex Robotics raised $20M in a Series A round. Investors in this round included Highland Ridge Capital ($12M) and Westport Ventures ($8M).

### Apex Robotics Seed — Jun 2018
In 06/01/2018, Apex Robotics raised $3M in a Seed round. Investors in this round included Highland Ridge Capital ($3M).

## Apex Robotics Current Investors
Highland Ridge Capital has been an investor in Apex Robotics since 06/01/2018. Status: Active. Holding: Minority.
Westport Ventures has been an investor in Apex Robotics since 01/15/2020. Status: Active. Holding: Minority.

## Apex Robotics Similar Companies and Competitors
RoboFlow (San Francisco, CA) is similar to Apex Robotics, ranked 1 in similarity.
AutoMech Systems (Boston, MA) is similar to Apex Robotics, ranked 2 in similarity.

Note how the entity name "Apex Robotics" appears on nearly every line. This is a critical pattern — see Best Practice #6 below.

Sectioning improves chunking, relationship inference, and grounding.

2. Submit Materialized Views via the Cashmere API

POST endpoint documentation.

Each submitted document should include:

A unique external_id
One or more collection_id values
Optional tags
Metadata (title, publisher, source_url)
The materialized HTML/Markdown as content

Suggested Fields

collection_id

Comma-delimited list of collection IDs representing the groups this content belongs to. Collections need to exist prior to upload.

Collections can be retrieved using the list_collections endpoint, which will return the ID’s of the collections and can be used in this field.

Example:

14,29,56

external_id

Your internal ID for this record.

If IDs overlap across tables (e.g., all are integers), prefix them for global uniqueness:

company_145726
investor_22291
deal_9081
fund_112

User Guides

Publisher Guides

Developer Guides