A key-value storage system designed to handle massive amounts of data with minimal latency. Whether you’re building a URL shortener, a content delivery system, or any application requiring quick data retrieval, Cloudflare KV Storage offers a powerful and flexible solution.

What is Cloudflare KV Storage? Link to heading

Cloudflare KV (Key-Value) Storage is a globally distributed, eventually consistent key-value store that allows developers to store and retrieve data across Cloudflare’s edge network.

Key Features of Cloudflare KV: Link to heading

  • Global Distribution: Data stored in KV is automatically replicated across Cloudflare’s global network, ensuring that it is close to users wherever they are.

  • Eventual Consistency: While KV is globally distributed, it offers eventual consistency, meaning that updates to a key might take a little time to propagate across all data centers. This is a trade-off for the performance and scalability it provides.

  • Low Latency: KV is optimized for read-heavy workloads, delivering fast read access to data.

  • Large Storage Capacity: It is designed to handle large volumes of data with millions of keys.

  • Integration with Workers: KV is tightly integrated with Cloudflare Workers, making it easy to use KV in serverless functions.

KV Storage vs. SQL/NoSQL Databases Link to heading

You might wonder how KV Storage compares to more traditional storage solutions like SQL or NoSQL databases. Here’s a quick overview:

SQL Databases Link to heading

  • Structure: Relational, with defined schemas.

  • Use Case: Best for applications requiring complex queries, transactions, and strong consistency.

  • Performance: Higher latency for global distribution.

NoSQL Databases Link to heading

  • Structure: Flexible, schema-less.

  • Use Case: Suitable for applications requiring horizontal scaling and handling large volumes of unstructured data.

  • Performance: Better for distributed systems but still may require more management than KV Storage.

KV Storage Link to heading

  • Structure: Simple key-value pairs.

  • Use Case: Ideal for caching, configuration, and scenarios with high read frequency and low write frequency.

  • Performance: Extremely low latency due to global distribution and eventual consistency.

    Source code Link to heading

    If you’d like to see the full source code for this project, it’s available on GitHub.

Setting Up KV Storage Link to heading

1. Install wrangler: Link to heading

wrangler is the CLI tool used to manage Cloudflare Workers projects.

npm install -g wrangler

If you don’t have Node.js installed, you’ll need to install it first, as wrangler is a Node.js package.

2. Login to Cloudflare: Link to heading

To link wrangler to your Cloudflare account, run:

wrangler login

This will open a browser window where you can authorize wrangler to access your Cloudflare account.

3. Create a New Workers Project: Link to heading

Create a new Cloudflare Workers project using wrangler:

wrangler init my-workers-project

This will create a new directory named my-workers-project with the necessary files and structure.

4. Set Up KV Namespace: Link to heading

  1. Log in to the Cloudflare dashboard.

  2. Go to Workers and then KV.

  3. Create a new KV namespace.

  4. Note the namespace’s ID.

5. Configure wrangler.toml: Link to heading

Open the wrangler.toml file in your project directory and add the KV namespace configuration:

name = "my-workers-project"
type = "javascript"

kv_namespaces = [
  { binding = "MY_KV", id = "your-kv-namespace-id" }
]

compatibility_date = "2023-08-01" # Adjust to current date
node_compat = true
nodejs_compat = true
workers_dev = true

Replace "your-kv-namespace-id" with the ID of the KV namespace you created.

6. Create the Worker Script: Link to heading

Replace the contents of the index.js or src/index.js file with the API script provided earlier (which supports GET, POST, PUT, and DELETE requests).

7. Run Your Worker Locally: Link to heading

Cloudflare Workers don’t run locally in the traditional sense, but you can use wrangler dev to simulate it. This command creates a local development server that mimics the behavior of the Workers platform:

wrangler dev

This will start the worker on a local server, usually available at http://localhost:8787.

8. Test Your API Locally: Link to heading

You can now use tools like curl, Postman, or a browser to test your API endpoints locally.

For example:

  • POST Request:

    curl -X POST http://localhost:8787/post \
  -H 'Content-Type: application/json' \
  -d '{"key": "someKey", "value": "someValue"}'

  • GET Request:

    bashCopy codecurl http://localhost:8787/get?key=someKey

  • PUT Request:

    curl -X PUT http://localhost:8787/put \
  -H 'Content-Type: application/json' \
  -d '{"key": "someKey", "value": "newValue"}'

  • DELETE Request:

    curl -X DELETE http://localhost:8787/delete \
  -H 'Content-Type: application/json' \
  -d '{"key": "someKey"}'

9. Deploy to Cloudflare: Link to heading

Once you’re satisfied with your local testing, you can deploy your worker to Cloudflare’s network:

wrangler publish

This will deploy your Worker to the Cloudflare network and make it accessible globally.

Here’s an example of a URL shortener with authentication using Cloudflare Workers Link to heading

1.1 Shorten URL Link to heading 

import { generateUniqueShortcode } from '../../utils/generateUniqueShortcode'

export async function handleCreateShortUrl(request, env) {
	const { originalUrl, customShortcode } = await request.json()

	let shortcode

	if (customShortcode) {
		const existingUrl = await env.MY_KV.get(`url:${customShortcode}`)
		if (existingUrl) {
			return new Response(
				'Shortcode already in use. Please choose another one.',
				{ status: 409 }
			)
		}
		shortcode = customShortcode
	} else {
		shortcode = await generateUniqueShortcode(env)
	}

	await env.MY_KV.put(
		`url:${shortcode}`,
		JSON.stringify({
			originalUrl,
			createdAt: new Date().toISOString(),
			clicks: 0,
		})
	)
	return new Response(
		JSON.stringify({
			shortUrl: `${new URL(request.url).origin}/go/${shortcode}`,
		}),
		{
			headers: { 'Content-Type': 'application/json' },
		}
	)
}

1.2 Get Original URL Link to heading 

export async function getOriginalUrl(shortcode, env) {
	const urlData = await env.MY_KV.get(`url:${shortcode}`, { type: 'json' })

	if (!urlData) {
		return new Response('Shortcode not found', { status: 404 })
	}
	urlData.clicks += 1
	await env.MY_KV.put(`url:${shortcode}`, JSON.stringify(urlData))
	if (urlData) {
		return new Response(urlData.originalUrl, {
			status: 302,
			headers: { 'Content-Type': 'application/json' },
		})
	} else {
		return new Response(JSON.stringify({ error: 'URL not found' }), {
			status: 404,
			headers: { 'Content-Type': 'application/json' },
		})
	}
}

import { handleCreateShortUrl } from './routes/urlShortener/createShortUrl';

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);

    // Route requests based on the path and method
    if (url.pathname === '/shorten' && request.method === 'POST') {
      // Call the function to handle creating a short URL
      return await handleCreateShortUrl(request, env, ctx);
    }

    // Default response for other routes
    return new Response('Not Found', { status: 404 });
  },
};

Approach: Maintain a User Index Link to heading

Whenever you create or add a new user, you also update a special key (e.g., userIndex) that stores a list of all user IDs. This way, you can retrieve the list of all users by simply fetching this index key.

Why Use Indexes? Link to heading

Since Cloudflare KV Storage is a simple key-value store without the ability to list or query keys directly, maintaining an index of keys (like user IDs or URL shortcodes) allows you to keep track of all the items of a certain type. This way, you can simulate querying or listing all users or URLs.

Step-by-Step Implementation Link to heading

1. Store User Data with Indexing Link to heading

When you add a new user to KV Storage, you also update the userIndex key with the user’s ID.

  • Adding a New User:
async function addUser(userId, userData) {
  // Store the user data
  await KV.put(`user:${userId}`, JSON.stringify(userData));

  // Update the user index
  let userIndex = JSON.parse(await KV.get("userIndex")) || [];
  userIndex.push(userId);
  await KV.put("userIndex", JSON.stringify(userIndex));
}

2. Retrieve All Users Link to heading

To get the list of all users, fetch the userIndex key and then retrieve each user’s data.

  • Fetching All Users:
async function getAllUsers() {
  // Fetch the user index
  let userIndex = JSON.parse(await KV.get("userIndex")) || [];

  // Retrieve user data for each userId
  let users = [];
  for (let userId of userIndex) {
    let userData = JSON.parse(await KV.get(`user:${userId}`));
    users.push(userData);
  }
  return users;
}

Source code Link to heading

If you’d like to see the full source code for this project, it’s available on GitHub.

Conclusion Link to heading

Cloudflare KV Storage is a robust tool for developers looking to build scalable, globally distributed applications with minimal complexity. Its ease of use, combined with the performance benefits of Cloudflare’s edge network, makes it an excellent choice for a wide range of use cases, from simple URL shorteners to complex data caching systems.