Preparing Structured Data for Ingestion

<aside> 🧭

Nav Links

</aside>

Updated Nov. 17, 2025

This article uses a Fictional Private Markets Intelligence Platform as an Example

Organizations with structured or semi-structured datasets—such as company directories, investment databases, deal trackers, or fund analytics products—should prepare their content for Cashmere by converting each entity into a single materialized article view and submitting it to the POST /omnipub endpoint with correct metadata.

This guide uses a fictional Private Markets Intelligence Platform as the running example. In this fictional platform, the core entities include:

Companies
Investors
Funds
Service Providers
Deals

These entities live in their own relational tables but are presented to users as rich, joined profile pages. Cashmere needs those unified profile pages—converted to HTML or Markdown—as the ingestable content units.

1. Materialize Each Entity Into a Single Article View

A. How relational data becomes a profile page

In a relational system, an entity exists as a single row in a table, but the profile page users see is constructed from many joins.

Example: A Company profile may include:

Company details (from the companies table)
Deals involving the company (from deals)
Investors in those deals (from investors)
Funds associated with those investors (from funds)
Service providers participating in related deals (from service_providers)
Press releases or relevant news (from news)

Similarly, an Investor profile includes:

Investor details
All deals the investor participated in
All companies tied to those deals
Any funds the investor manages

These views naturally contain cross-linked content and repeated rows.

This repetition is correct and required.

B. Repeat data everywhere it appears

When you materialize the profile into an article view:

Do not de-duplicate entities
Do not reference other entities indirectly
Do not assume “the LLM can fetch that elsewhere”

Example:

If an investor made three separate investments into a company across three rounds, the Company profile should list:
- That investor three separate times, each with the correct round and amount
On the Investor profile, the same company may appear multiple times if they participated in multiple deals
On the Deal profile, show all participants, even if they appear in other profiles

This gives Cashmere a complete, self-contained, text-based representation of the relationships that exist in your dataset.

C. Remove internal/self-referential links

Your application UI likely has links to other profiles (e.g., clicking a Company name takes you to the Company profile).

For Cashmere ingestion:

Remove all internal hyperlinks
Replace them with plain text
Only keep URLs that point to external sources (e.g., news articles, filings)

Internal links confuse LLM interpretation of the content and undermine the goal of directing users to your canonical source_url.

D. Output the profile as HTML or Markdown with clear sections

Use headings to separate logical segments:

# Apex Robotics — Company Profile

## Overview
- Headquarters: Seattle, WA
- Founded: 2014
- Industry: Robotics

## Deal Activity
### Series A — Jan 2020
- Investors:
  - Highland Ridge Capital — $12M
  - Westport Ventures — $8M

### Seed — Jun 2018
- Investors:
  - Highland Ridge Capital — $3M

## Investors
- Highland Ridge Capital
- Westport Ventures

## Related News
- “Apex Robotics Launches Autonomous Drone Platform” — TechWire (2024)

Sectioning improves chunking, relationship inference, and grounding.

2. Submit Materialized Views via the Cashmere API

POST endpoint documentation.

Each submitted document should include:

A unique external_id
One or more collection_id values
Optional tags
Metadata (title, publisher, source_url)
The materialized HTML/Markdown as content

Suggested Fields

collection_id

Comma-delimited list of collection IDs representing the groups this content belongs to. Collections need to exist prior to upload.

Collections can be retrieved using the list_collections endpoint, which will return the ID’s of the collections and can be used in this field.

Example:

14,29,56

external_id

Your internal ID for this record.

If IDs overlap across tables (e.g., all are integers), prefix them for global uniqueness:

company_145726
investor_22291
deal_9081
fund_112

User Guides

Publisher Guides

Developer Guides