Working with Over 5,000 Records in Base44

Introduction

When a Base44 system grows beyond 5,000 records, strange symptoms start to appear. Pages display partial data, searches fail to find existing records, and relationships between Entities break for no apparent reason.

The reason is always the same: a single API call returns a maximum of 5,000 records.

In this article we'll cover the most common problems, the exact solutions, and copy-paste prompts you can use directly in Builder.

---

Understanding Limit & Skip

The two fundamental parameters in every API call:

| Parameter | What it does | Example | |-----------|-------------|----------| | Limit | How many records to return | 50 | | Skip | How many records to skip | 100 | | Fields | Which fields to return | ['id', 'name'] |

jsCopy
base44.entities.Leads.filter(
  query,           // filter condition
  '-created_date', // sort
  50,              // limit — how many records
  100,             // skip — where to start
  ['id', 'name', 'email', 'status'] // fields — which to return
)

Pagination formula:

jsCopy
skip = (currentPage - 1) * limit

// Page 1: limit 50, skip 0
// Page 2: limit 50, skip 50
// Page 3: limit 50, skip 100

---

Problem 1 — Incorrect Count Over 5,000

Symptom

The system shows "5,000 records" even though there are 9,800.

Cause

jsCopy
// ❌ Doesn't work above 5,000:
const leads = await Leads.filter(query, sort, 5000, 0)
const total = leads.length // Returns 5,000 — incorrect!

Fix

jsCopy
// ✅ Load in Batches:
const batch1 = await Leads.filter(query, sort, 5000, 0)
const batch2 = await Leads.filter(query, sort, 5000, 5000)
const batch3 = await Leads.filter(query, sort, 5000, 10000)

const total = batch1.length + batch2.length + batch3.length
// = 9,800 ✅

Builder Prompt

codeCopy
This entity has more than 5000 records and the count
is wrong. Implement multi-batch loading:
- Batch 1: limit 5000, skip 0
- Batch 2: limit 5000, skip 5000
- Batch 3: limit 5000, skip 10000
Stop fetching when a batch returns less than 5000 results.
Combine all results for correct total count.

---

Problem 2 — Broken Relationships Between Entities

Symptom

Client #7,500 can't find its associated lead.

Cause

jsCopy
// ❌ Scans all leads — stops at 5,000:
const lead = await Leads.filter({
  client_id: clientId
})
// Lead #7,500 is never found!

Fix

jsCopy
// ✅ Fetch directly by ID — always works:
if (client.lead_id) {
  const lead = await Leads.get(client.lead_id)
}

Builder Prompt

codeCopy
This page depends on both [ENTITY_A] and [ENTITY_B].
Records beyond position 5000 are not found.

Fix by fetching associated records directly by ID:
base44.entities.[ENTITY_B].get(record.[ENTITY_B]_id)

NEVER filter through all records to find associations.
Add null check: if (record.[ENTITY_B]_id) { fetch it }

---

Problem 3 — Slow Page Load

Symptom

The page takes 3+ seconds to load.

How to Diagnose

Open F12 → Network Tab, refresh the page, and look for:

| What to look for | Healthy | Problem | |-----------------|---------|----------| | Number of calls | Under 10 | Over 50 | | Response size | Under 50 kB | Over 100 kB | | Load time | Under 500ms | Over 1,000ms | | 404 errors | None | Present |

Fix — Priority Loading

jsCopy
// PHASE 1 — Immediate (under 1 second):
const leads = await Leads.filter(
  query,
  '-created_date',
  50,  // limit
  0,   // skip
  ['id', 'first_name', 'last_name',
   'phone', 'email', 'status'] // only needed fields!
)

// PHASE 2 — Background after render:
setTimeout(async () => {
  const stats = await loadStats()
  const filters = await loadFilters()
}, 0)

Builder Prompt

codeCopy
The [PAGE_NAME] page is slow. Fix with PRIORITY LOADING:

PHASE 1 - Show immediately (under 1 second):
- Fetch ONLY 50 records with limited fields:
  base44.entities.[ENTITY].filter(
    query, sort, 50, 0,
    ['id', 'field1', 'field2', 'field3']
  )

PHASE 2 - Load in background after render:
- Total count, statistics, filters

Maximum 5 API calls on initial page load.

---

Problem 4 — Too Many API Calls

Symptom

Network Tab shows 50–300+ calls on a single page load.

Cause

jsCopy
// ❌ Separate call per record:
clients.forEach(async (client) => {
  const lead = await Leads.get(client.lead_id)
})
// = 50 calls instead of 1!

Fix — Batch Loading

jsCopy
// ✅ One call for all:
const leadIds = clients.map(c => c.lead_id).filter(Boolean)
const leads = await Leads.filter(
  { id: { $in: leadIds } },
  null,
  leadIds.length
)

Builder Prompt

codeCopy
The page makes [NUMBER] API calls on load - too many.
Fix with batch loading instead of individual calls:

WRONG:
records.forEach(r => Entity.get(r.associated_id))

CORRECT:
const ids = records.map(r => r.associated_id).filter(Boolean)
const results = await Entity.filter(
  { id: { $in: ids } },
  null,
  ids.length
)

Goal: Maximum 5 API calls on initial page load.

---

Problem 5 — 404 Errors in Network Tab

Symptom

Red entries in Network Tab — records that can't be found.

Cause

The code is trying to load records that have been deleted.

Fix

jsCopy
// ✅ Always check before fetching:
if (record.associated_id) {
  try {
    const data = await Entity.get(record.associated_id)
    if (data) {
      // process the data
    }
  } catch (error) {
    // Show placeholder instead of crashing
    console.log('Record not found:', record.associated_id)
  }
}

Builder Prompt

codeCopy
Some records return 404 in Network tab.
Associated IDs no longer exist in [ENTITY_NAME].

Fix by:
1. Add null check: if (record.associated_id) { fetch it }
2. Wrap in try/catch to handle missing records
3. Show placeholder "N/A" instead of crashing
4. Don't block the rest of the page from loading

---

Summary — The Golden Rules

1. If you have an ID — fetch by ID. Never scan.

2. Limit = how many. Skip = where from. Fields = what.

3. Fewer kB = faster load — only request what you need.

4. F12 → Network Tab is your best diagnostic friend.

5. Over 5,000 records? Split into Batches of 5,000.

---

Have questions or specific issues? Share them in the comments 👇