Introduction

It is often easy to fetch, load, and render a few KBs of Geolocation data on a map. It doesn’t matter what the data is (polygon/lines/points), or what the library of the map is (Google Maps, ESRI, OpenLayers, MapBox, Leaflet). Even if it requires frequent API calls to a remote server, it is fine. No brainer.

However, when we’re talking millions of data, consuming MBs of bandwidth on network load, we want a more optimized approach. As with larger payloads, both the user and the server are going to take a hit.

So, how do we solve this? We have quite a few battle-tested solutions:

• Caching/versioning
• Smoothing (cartography)

Why use IndexedDB?

Today we’ll see how to fix this with caching. Now with caching, we have quite a few options on the browser:

• Session storage
• Local storage
• IndexedDB

Session storage: We could use Session storage, but the problem with this would be that we can only retain the data until a session lasts. We’d still need to make a remote API call to get the data again.

Local storage: We could use Local storage but it comes with a very limited capacity. We cannot store data if it exceeds 10MB, in some browsers even less. We could save some space by stringifying the data and storing them, however, the process of stringifying and un-stringifying the data would be a tradeoff between time and space.

IndexedDB: So we try IndexedDB. It practically has no size limit. That is, it’ll depend on the client’s disk space where the browser is installed.

So the idea of using IndexedDB is:

• We try to get some resources, say geofences (polygons, set of coordinates data) from a server
• We check if the data is already available in our local database (IndexedDB)
• If it is available, we serve the data from the local DB
• If it is unavailable, we store data in the local DB by making an API request in the remote server, and then server from the local DB (if we want, we can do it parallelly as well for the first API call)
• Next time we try to get the data, it is served from the local database
• We can create a service worker in the background, which could update the data in our local database in realtime whenever some updates are made (will create a separate article for that)

Implementation

Let’s first create a service for the IndexedDB which we’d use to fetch and store data. We create a file named IndexedDbService.ts. Here’s the file content:

class IndexedDBManager {
  private dbName: string;

  constructor(dbName: string) {
    this.dbName = dbName;
  }

  private openDatabase(objectStoreName: string): Promise<IDBDatabase> {
    return new Promise((resolve, reject) => {
      try {
        const clientDbRequest = indexedDB.open(this.dbName, 1);

        clientDbRequest.onupgradeneeded = (event: any) => {
          const db = event.target.result;
          db.createObjectStore(objectStoreName, { keyPath: 'id' });
        };

        clientDbRequest.onsuccess = (event: any) => {
          const db = event.target.result;
          resolve(db);
        };

        clientDbRequest.onerror = (event: any) => {
          reject(event.target.error);
        };
      } catch (e) {
        console.log(e);
        reject(e);
      }
    });
  }

  public saveToIndexedDB(data: any, objectStoreName: string): Promise<void> {
    return this.openDatabase(objectStoreName).then((db: IDBDatabase) =>
      new Promise<void>((resolve, reject) => {
        try {
          const transactionContext = db.transaction(new Array(objectStoreName), 'readwrite');
          const objectStore = transactionContext.objectStore(objectStoreName);

          data.forEach((objectData: any) => {
            objectStore.put(objectData);
          });

          transactionContext.oncomplete = () => {
            console.log('Data written to IndexedDB');
            resolve();
          };

          transactionContext.onerror = (event: any) => {
            console.error('Error writing to IndexedDB', event.target.error);
            reject(event.target.error);
          };
        } catch (e) {
          console.log(e);
          reject(e);
        }
      })
    );
  }


  public getFromIndexedDB(objectStoreName: string, readWriteMode: IDBTransactionMode = 'readonly'): Promise<any[]> {
    return new Promise((resolve, reject) => {
      try {
        this.openDatabase(objectStoreName).then((db: IDBDatabase) => {
          const transactionContext = db.transaction(new Array(objectStoreName), readWriteMode);
          const transactionObjectStore = transactionContext.objectStore(objectStoreName);
          const request = transactionObjectStore.getAll();

          request.onsuccess = (event: any) => {
            const data = event.target.result;
            resolve(data);
          };

          request.onerror = (event: any) => {
            reject(event.target.error);
          };
        });
      } catch (e) {
        console.log(e);
        reject(e);
      }
    });
  }

  public isDataInIndexedDB(objectStoreName: string): Promise<boolean> {
    return new Promise((resolve, reject) => {
      try {
        this.getFromIndexedDB(objectStoreName).then((savedData: any) => {
          resolve(savedData.length > 0);
        }).catch((error) => {
          reject(error);
        });
      } catch (e) {
        console.log(e);
        reject(e);
      }
    });
  }
}

export default IndexedDBManager;Code language: JavaScript (javascript)

In the service, we write the following methods:

• openDatabase: To open / create a database with a given name
• saveToIndexedDB: To save data to DB with some data and a given object store name
• getFromIndexedDB: To retrieve data from DB
• isDataInIndexedDB: To check if we already have the data in the DB

Now, we can use the service in an action or a method as we want:

import IndexedDBManager from './IndexedDbService.ts';

async getPolygonsList(storeName?: string, mapper?: (dto: any) => any) {
    const objectStoreName = 'geofences';

    const fetchGeofencesData = async () => {
      // here you make your remote api call and return response
      // you can define you own remote calls as you wish
      const polygonsApi = new PolygonsApiRequest();
      const polygonsListStore = usePaginationStore(POLYGONS_PAGINATION_MODULE);
      const data = await polygonsListStore.handleAPICall<Polygon>(polygonsApi.getPolygons.bind(polygonsApi), 
                    mapper ?? GeofenceMapper.mapToGeofence);

      return data;
    }

    const indexedDBManager = new IndexedDBManager('geofencesDB');

    const result = await indexedDBManager.isDataInIndexedDB(objectStoreName).then(async (dataExists: any) => {
      let polygons;

      if (dataExists) {
        await indexedDBManager.getFromIndexedDB(objectStoreName).then((storedData: any) => {
          console.log('Returning data from indexedDB...');
          // update your states here from the DB here
          const polygonsListStore = usePaginationStore(POLYGONS_PAGINATION_MODULE);
          polygonsListStore.data = storedData;
          polygons = storedData;
        })
      } else {
        const data: any = await fetchGeofencesData();
        indexedDBManager.saveToIndexedDB(data, objectStoreName);
        polygons = data;
      }
      return polygons;
    });

    return result;
  },Code language: JavaScript (javascript)

In the above file, when getPolygonsList() is called, the following flow happens:

• We call getPolygonsList() to get the list of polygons (to render them on a map)
• We call indexedDBManager.isDataInIndexedDB() to check if the data is already available in the database
• If it is available, we call indexedDBManager.getFromIndexedDB() to get the data and store it in the temporary polygons array
• If it is unavailable, we call fetchGeofencesData() to fetch the data first and then save them to the DB by calling indexedDBManager.saveToIndexedDB() and at the same time return the data. This is to fulfill the Promise<T> of getPolygonsList().

With this approach, I was able to load and render data of 22.8MBs in 2.3s, which would otherwise take 1 min 21s.