Backend Development: Full lesson content. # Express.js **Express.js** is a minimal web framework for **Node.js**. It gives you a clean way to: * define **routes** (URLs) * add **middleware** (code that runs before/after routes) * send responses (HTML, JSON, files) ## Install [Section titled “Install”](#install) ```bash npm install express ``` ## Your First Server [Section titled “Your First Server”](#your-first-server) ```js import express from 'express'; const app = express(); const port = 3000; app.get('/', (req, res) => { res.send('Hello from Express!'); }); app.listen(port, () => { console.log(`Server running at http://localhost:${port}`); }); ```  ## Routing (The Core Concept) [Section titled “Routing (The Core Concept)”](#routing-the-core-concept) A **route** matches a request by: * HTTP method (GET/POST/PUT/PATCH/DELETE) * path (like `/users`) Examples: ```js app.get('/health', (req, res) => res.json({ ok: true })); app.post('/users', (req, res) => { res.status(201).json({ message: 'User created' }); }); ``` ### Route Parameters (`:id`) [Section titled “Route Parameters (:id)”](#route-parameters-id) Use route params when you want to address **one specific resource**. ```js app.get('/users/:id', (req, res) => { const { id } = req.params; res.json({ id, name: 'Example' }); }); ``` Request: `GET /users/42` `req.params`: ```json { "id": "42" } ``` ### Query Parameters (`?search=...`) [Section titled “Query Parameters (?search=...)”](#query-parameters-search) Use query params for **filtering, sorting, pagination**. ```js app.get('/users', (req, res) => { const { role, page = '1' } = req.query; res.json({ role, page }); }); ``` Request: `GET /users?role=admin&page=2` ## Request / Response Objects [Section titled “Request / Response Objects”](#request--response-objects) Every route handler receives: * `req` (request): incoming data * `res` (response): what you send back Common things you use: ```js app.get('/info', (req, res) => { res.json({ method: req.method, path: req.path, headers: req.headers, }); }); ``` ## JSON APIs (Reading the Request Body) [Section titled “JSON APIs (Reading the Request Body)”](#json-apis-reading-the-request-body) To read JSON bodies, you need the JSON middleware: ```js app.use(express.json()); app.post('/echo', (req, res) => { res.json({ youSent: req.body }); }); ``` ## Middleware [Section titled “Middleware”](#middleware) **Middleware** is a function that runs during request processing. It can: * log requests * check auth * attach data to `req` * stop the request with a response * or pass control to the next handler ### Logging Middleware Example [Section titled “Logging Middleware Example”](#logging-middleware-example) ```js app.use((req, res, next) => { console.log(`${req.method} ${req.path}`); next(); }); ```  ### Route-Specific Middleware Example [Section titled “Route-Specific Middleware Example”](#route-specific-middleware-example) ```js function requireApiKey(req, res, next) { if (req.headers['x-api-key'] !== 'secret') { return res.status(401).json({ error: 'Unauthorized' }); } next(); } app.get('/private', requireApiKey, (req, res) => { res.json({ secretData: 123 }); }); ``` ## Routers (Organizing Routes) [Section titled “Routers (Organizing Routes)”](#routers-organizing-routes) As your API grows, you usually group routes into a **router**. Example: ```js import express from 'express'; const usersRouter = express.Router(); usersRouter.get('/', (req, res) => res.json([{ id: 1, name: 'Alice' }])); usersRouter.get('/:id', (req, res) => res.json({ id: req.params.id })); export default usersRouter; ``` Then mount it: ```js import usersRouter from './routes/users.js'; app.use('/users', usersRouter); ``` This creates: * `GET /users` * `GET /users/:id` ## Status Codes (Quick Examples) [Section titled “Status Codes (Quick Examples)”](#status-codes-quick-examples) ```js res.status(200).json({ ok: true }); res.status(201).json({ created: true }); res.status(404).json({ error: 'Not found' }); ``` ## Error Handling (Simple Pattern) [Section titled “Error Handling (Simple Pattern)”](#error-handling-simple-pattern) Express treats a function with **four arguments** as an error handler. ```js app.get('/boom', (req, res) => { throw new Error('Something broke'); }); app.use((err, req, res, next) => { console.error(err); res.status(500).json({ error: 'Internal Server Error' }); }); ``` ## Serving Static Files [Section titled “Serving Static Files”](#serving-static-files) If you have a folder like `public/`, you can serve files from it: ```js app.use(express.static('public')); ``` Then `public/logo.png` becomes: `GET /logo.png` ## Where Express Is Used [Section titled “Where Express Is Used”](#where-express-is-used) * REST APIs for web and mobile apps * Backend-for-frontend (BFF) servers * Prototypes and small-to-medium backends (and sometimes parts of larger systems) Industry examples (companies using Node/Express-style backends): * **Netflix**: has used Node.js widely for web services * **PayPal**: early well-known Node.js adoption for web apps * **Uber**: uses Node.js for various services ## Related Frameworks (Mentions) [Section titled “Related Frameworks (Mentions)”](#related-frameworks-mentions) * **NestJS**: A more structured Node.js framework (TypeScript-first) inspired by Angular. ## References [Section titled “References”](#references) * [Express.js Official Site](https://expressjs.com/) * [Express API Reference](https://expressjs.com/en/4x/api.html) # Real-Time Data Real-time features let the server push updates to clients without waiting for the next normal page/API refresh. Typical examples: * Chat messages * Live dashboards (stocks, sensors) * Notifications * Multiplayer collaboration ## The Core Problem [Section titled “The Core Problem”](#the-core-problem) HTTP request/response is usually **client-driven**:  For real-time updates, we want **server-driven** delivery:  ## Long Polling [Section titled “Long Polling”](#long-polling) With **long polling**, the client sends a request and the server **holds it open** until there is new data (or a timeout), then the client immediately starts a new request. * **Pros**: Works almost everywhere, simple mental model. * **Cons**: More overhead (many HTTP requests), can get expensive at scale. ### Where It’s Used [Section titled “Where It’s Used”](#where-its-used) * When you need near-real-time updates but can’t rely on WebSockets (network restrictions, old infrastructure) * Simple bots / integrations where “poll for updates” is acceptable ### Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies) * **Telegram**: Bot API supports long polling to receive updates  ### Express Integration Example [Section titled “Express Integration Example”](#express-integration-example) This is a minimal pattern: keep a list of “waiting” HTTP responses and complete them when an update happens. **Server (Express):** ```js import express from 'express'; const app = express(); app.use(express.json()); // For demo purposes we keep waiting clients in memory. // In real apps, updates usually come from a DB change, queue, or pub/sub. let waiting = []; app.get('/updates', (req, res) => { // Hold the request open. req.setTimeout(30_000); // If no update arrives, return a "no update" response. const timeoutId = setTimeout(() => { waiting = waiting.filter((entry) => entry.res !== res); res.status(204).end(); }, 25_000); waiting.push({ res, timeoutId }); }); // Demo endpoint to trigger an update. app.post('/publish', (req, res) => { const update = { message: req.body?.message ?? 'Hello from server', at: new Date().toISOString(), }; const clients = waiting; waiting = []; for (const { res: clientRes, timeoutId } of clients) { clearTimeout(timeoutId); clientRes.json(update); } res.status(202).json({ deliveredTo: clients.length }); }); app.listen(3000, () => console.log('http://localhost:3000')); ``` **Client (browser):** ```js async function pollForever() { while (true) { const res = await fetch('/updates', { cache: 'no-store' }); if (res.status === 204) continue; // no update, immediately poll again const data = await res.json(); console.log('update', data); } } pollForever(); ``` ## Server-Sent Events (SSE) [Section titled “Server-Sent Events (SSE)”](#server-sent-events-sse) **SSE** is a one-way stream: **server → client** over a single long-lived HTTP connection. * **Pros**: Great for streams of updates (dashboards, notifications), simpler than WebSockets. * **Cons**: One-way only; client still needs normal HTTP requests to send data. ### Where It’s Used [Section titled “Where It’s Used”](#where-its-used-1) * Live dashboards and notification feeds * Streaming text responses (server continuously sends chunks) ### Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies-1) * **OpenAI**: streaming API responses commonly use SSE  ### Express Integration Example [Section titled “Express Integration Example”](#express-integration-example-1) SSE is just an HTTP response that stays open and streams text lines. **Server (Express):** ```js import express from 'express'; const app = express(); app.use(express.json()); const clients = new Set(); app.get('/events', (req, res) => { res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); res.flushHeaders?.(); // Optional: an initial message so the client knows it's connected. res.write(`event: ready\ndata: ${JSON.stringify({ ok: true })}\n\n`); clients.add(res); const heartbeat = setInterval(() => { // Comment line keeps some proxies from timing out the connection. res.write(': keep-alive\n\n'); }, 15_000); req.on('close', () => { clearInterval(heartbeat); clients.delete(res); }); }); // Demo endpoint to broadcast an event. app.post('/publish', (req, res) => { const payload = { message: req.body?.message ?? 'Hello SSE', at: new Date().toISOString(), }; for (const clientRes of clients) { clientRes.write(`event: message\ndata: ${JSON.stringify(payload)}\n\n`); } res.status(202).json({ deliveredTo: clients.size }); }); app.listen(3000, () => console.log('http://localhost:3000')); ``` **Client (browser):** ```js const events = new EventSource('/events'); events.addEventListener('message', (e) => { console.log('message', JSON.parse(e.data)); }); ``` ## WebSockets [Section titled “WebSockets”](#websockets) **WebSockets** upgrade an HTTP connection into a persistent, **two-way** channel. * **Pros**: Best for interactive real-time (chat, collaboration), low latency. * **Cons**: More complex to deploy (proxies/load balancers), requires careful auth and scaling. ### Where It’s Used [Section titled “Where It’s Used”](#where-its-used-2) * Chat and messaging * Multiplayer / collaborative apps (shared cursors, live edits) * Real-time trading / live data feeds ### Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies-2) * **Slack**: real-time messaging and updates rely on persistent connections * **Discord**: real-time chat and presence relies on persistent connections  ### Express Integration Example [Section titled “Express Integration Example”](#express-integration-example-2) Express doesn’t “do” WebSockets by itself. A common approach is to use Express for HTTP routes and attach a WebSocket server to the same underlying HTTP server. **Server (Express + ws):** ```js import express from 'express'; import http from 'http'; import { WebSocketServer } from 'ws'; const app = express(); const server = http.createServer(app); app.get('/health', (req, res) => res.json({ ok: true })); const wss = new WebSocketServer({ server }); wss.on('connection', (socket) => { socket.send( JSON.stringify({ type: 'welcome', at: new Date().toISOString() }) ); socket.on('message', (raw) => { // Broadcast to everyone (simple chat-style demo) for (const client of wss.clients) { if (client.readyState === 1) client.send(raw); } }); }); server.listen(3000, () => console.log('http://localhost:3000')); ``` **Client (browser):** ```js const ws = new WebSocket('ws://localhost:3000'); ws.addEventListener('message', (e) => { console.log('ws message', e.data); }); ws.addEventListener('open', () => { ws.send(JSON.stringify({ type: 'chat', text: 'hello' })); }); ``` ## Choosing the Right Approach [Section titled “Choosing the Right Approach”](#choosing-the-right-approach) * **Long polling**: simplest fallback when you can’t keep a connection. * **SSE**: best when the server mostly pushes updates and clients rarely send messages. * **WebSockets**: best when both sides frequently send messages. ## HTTP Streaming (Chunked Responses) [Section titled “HTTP Streaming (Chunked Responses)”](#http-streaming-chunked-responses) “HTTP streaming” usually means the server returns a normal HTTP response, but instead of sending it all at once, it **streams the body in chunks over time**. This is the idea behind many “streaming AI responses” and large downloads: * The client makes **one** request. * The server starts responding immediately. * More data arrives over time until the server ends the response. ### How This Relates to SSE [Section titled “How This Relates to SSE”](#how-this-relates-to-sse) * **SSE** is a specific *format* for streaming over HTTP (`text/event-stream`) with named events and automatic reconnection support in the browser (`EventSource`). * **HTTP streaming** is more general: you can stream plain text, JSON lines, or any custom format. Common streaming formats: * **Plain text chunks** (`text/plain`): simplest for incremental text. * **NDJSON / JSON Lines** (`application/x-ndjson`): one JSON object per line (good for parsing incrementally). ### When It’s Used [Section titled “When It’s Used”](#when-its-used) * Streaming logs or progress updates * Streaming long-running job results as they’re produced * Token-by-token / chunk-by-chunk text generation ### Key Practical Notes [Section titled “Key Practical Notes”](#key-practical-notes) * Many reverse proxies buffer responses by default. If you expect “live” streaming, you often need to disable buffering at the proxy/load balancer level. * Keep-alives and timeouts matter: long streams can be cut off by infrastructure if you don’t send data periodically. * Streaming is still “HTTP”: it’s great for **server → client** data, but doesn’t replace **bi-directional** protocols like WebSockets. ### Express Integration Example (Text Chunks) [Section titled “Express Integration Example (Text Chunks)”](#express-integration-example-text-chunks) **Server (Express):** ```js import express from 'express'; const app = express(); app.get('/stream', async (req, res) => { res.setHeader('Content-Type', 'text/plain; charset=utf-8'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); res.flushHeaders?.(); for (let i = 1; i <= 5; i++) { res.write(`chunk ${i}\n`); await new Promise((r) => setTimeout(r, 500)); } res.end('done\n'); }); app.listen(3000, () => console.log('http://localhost:3000')); ``` **Client (browser using `fetch` streaming):** ```js const res = await fetch('/stream'); const reader = res.body.getReader(); const decoder = new TextDecoder(); while (true) { const { value, done } = await reader.read(); if (done) break; console.log(decoder.decode(value, { stream: true })); } ``` ## Security Notes [Section titled “Security Notes”](#security-notes) * Use **TLS** (`https://` and `wss://`) so data is encrypted in transit. * Authenticate the connection (e.g., cookie session or token) and enforce **authorization** on every message/channel. * Validate input on real-time channels the same way you validate REST inputs. # API Styles **API** stands for **Application Programming Interface**. It allows different software systems to talk to each other. This section is split into smaller pages: * [REST API](/backend/api/styles/rest) * [OpenAPI (Swagger)](/backend/api/styles/openapi) * [GraphQL](/backend/api/styles/graphql) * [gRPC](/backend/api/styles/grpc) * [SOAP](/backend/api/styles/soap) ## How to Choose [Section titled “How to Choose”](#how-to-choose) * **REST**: best default for most teams and public APIs * **OpenAPI**: best when you want strong docs/contract tooling around HTTP APIs * **GraphQL**: best when clients need flexible shapes of data * **gRPC**: best for internal services that want strong typing + performance * **SOAP**: mostly for legacy enterprise integrations ## References [Section titled “References”](#references) * [RESTful API Design](https://restfulapi.net/) * [MDN HTTP Methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods) # GraphQL **GraphQL** is an API style where the client can request *exactly* the fields it needs. Instead of multiple endpoints (`/users`, `/users/:id`, `/users/:id/posts`), GraphQL often exposes a single endpoint (commonly `/graphql`) and the client sends queries. ## Where GraphQL Is Used [Section titled “Where GraphQL Is Used”](#where-graphql-is-used) * Frontends that need flexible queries (complex pages, many UI components) * Public APIs where clients need different shapes of the same data ## Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies) * **GitHub**: GraphQL API for developer/platform integrations * **Shopify**: GraphQL Admin API * **Meta (Facebook)**: created GraphQL and uses it heavily ## Pros / Cons [Section titled “Pros / Cons”](#pros--cons) Pros: * Reduces over-fetching/under-fetching (client chooses fields) * Strong typing via schema * Great developer experience with tooling Cons: * Caching can be harder than REST (queries vary) * You still need to design your schema carefully * Performance pitfalls (e.g., N+1 queries) if resolvers are naive ## Tiny Example [Section titled “Tiny Example”](#tiny-example) Query: ```graphql query { user(id: 42) { id name posts { id title } } } ``` Response: ```json { "data": { "user": { "id": 42, "name": "Alice", "posts": [{ "id": 1, "title": "Hello" }] } } } ``` ## References [Section titled “References”](#references) * [GraphQL](https://graphql.org/) # gRPC **gRPC** is a high-performance RPC framework that commonly uses HTTP/2 + Protocol Buffers. It’s most often used for **service-to-service** communication inside a company (microservices). ## Where gRPC Is Used [Section titled “Where gRPC Is Used”](#where-grpc-is-used) * Internal microservice-to-microservice communication * Low-latency APIs with strongly typed contracts ## Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies) * **Google**: created gRPC and uses it extensively internally * **Uber**: widely uses gRPC for internal services ## Pros / Cons [Section titled “Pros / Cons”](#pros--cons) Pros: * Fast (binary protocol + HTTP/2) * Strong typing and code generation * Supports streaming (client streaming, server streaming, bi-directional) Cons: * Browser support is more complex than plain REST * Debugging by hand is harder than JSON APIs ## Tiny Example (Proto) [Section titled “Tiny Example (Proto)”](#tiny-example-proto) ```proto syntax = "proto3"; service Users { rpc GetUser (GetUserRequest) returns (User); } message GetUserRequest { int32 id = 1; } message User { int32 id = 1; string name = 2; } ``` ## References [Section titled “References”](#references) * [gRPC](https://grpc.io/) # OpenAPI (Swagger) **OpenAPI** is a standard format (JSON/YAML) for describing an HTTP API: endpoints, parameters, request/response shapes, and authentication. It’s often shown with **Swagger UI**. ## Where OpenAPI Is Used [Section titled “Where OpenAPI Is Used”](#where-openapi-is-used) * API documentation portals * Generating client SDKs (frontend/mobile) from a single spec * Contract-first development between teams (backend/frontend) ## Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies) * **GitHub**: publishes an OpenAPI description for its REST API * **Azure / Microsoft**: heavy usage of API specs and generated SDKs across cloud services ## Why Teams Like It [Section titled “Why Teams Like It”](#why-teams-like-it) * One source of truth for routes + payloads * Makes breaking changes easier to spot * Helps new developers explore APIs quickly ## Tiny Example (YAML) [Section titled “Tiny Example (YAML)”](#tiny-example-yaml) ```yaml openapi: 3.0.0 info: title: Example API version: 1.0.0 paths: /users: get: summary: List users responses: "200": description: OK /users/{id}: get: summary: Get one user parameters: - name: id in: path required: true schema: type: integer responses: "200": description: OK ``` ## Swagger vs OpenAPI [Section titled “Swagger vs OpenAPI”](#swagger-vs-openapi) * **OpenAPI** is the specification. * **Swagger UI** is a popular tool that renders an interactive docs website from an OpenAPI file. ## References [Section titled “References”](#references) * [OpenAPI Specification](https://spec.openapis.org/oas/latest.html) * [Swagger UI](https://swagger.io/tools/swagger-ui/) # REST API **REST** (Representational State Transfer) is the most common style for building web APIs. It treats data (users, posts, products) as **resources** and uses standard HTTP methods. ## Where REST Is Used [Section titled “Where REST Is Used”](#where-rest-is-used) * Public web APIs (payments, shipping, maps, social media) * Internal service APIs inside a company * Mobile app backends (iOS/Android calling HTTP endpoints) ## Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies) * **GitHub**: REST API for repos, issues, actions * **Stripe**: REST API for payments and billing * **Shopify**: REST Admin APIs (alongside GraphQL) ## Resources (The “Nouns”) [Section titled “Resources (The “Nouns”)”](#resources-the-nouns) A **resource** is a thing your API exposes. Examples: * `users` * `posts` * `products` * `orders` In REST, URLs usually represent **resources** (nouns), and the HTTP method represents the **action** (verb). Good: * `GET /users` (get a list of users) * `GET /users/42` (get user 42) Avoid: * `GET /getUsers` * `POST /createUser` ## Key Principles [Section titled “Key Principles”](#key-principles) 1. **Stateless**: the server doesn’t remember client state between requests. Each request must include what it needs (like an auth token). 2. **Client-Server**: frontend (client) and backend (server) are separate and communicate via HTTP. 3. **Uniform Interface**: resources are identified by URLs (e.g., `/users/1`). ## HTTP Methods [Section titled “HTTP Methods”](#http-methods) * **GET**: retrieve data (read) * **POST**: create new data (create) * **PUT**: replace existing data (all fields) * **PATCH**: update existing data (some fields) * **DELETE**: remove data ## Routing (Endpoints) [Section titled “Routing (Endpoints)”](#routing-endpoints) **Routing** maps an incoming request (method + URL) to a handler. An **endpoint** is a specific route, like `GET /users/42`. Most REST APIs follow a predictable route pattern: | Resource operation | HTTP | Route | | -------------------- | -----: | ------------ | | List resources | GET | `/users` | | Create resource | POST | `/users` | | Get one resource | GET | `/users/:id` | | Replace one resource | PUT | `/users/:id` | | Update one resource | PATCH | `/users/:id` | | Delete one resource | DELETE | `/users/:id` | Where `:id` is a **route parameter** (a placeholder inside the URL). ## Query Parameters (Filtering, Sorting, Pagination) [Section titled “Query Parameters (Filtering, Sorting, Pagination)”](#query-parameters-filtering-sorting-pagination) Query params come after `?` and are great for **searching and listing**. Examples: * `GET /users?role=admin` (filter) * `GET /users?sort=-createdAt` (sort descending) * `GET /users?page=2&limit=20` (pagination) Rule of thumb: * Use **route params** for identifying a single resource: `/users/42` * Use **query params** for modifying a collection query: `/users?role=admin` ## Nested Resources [Section titled “Nested Resources”](#nested-resources) Some resources belong to others. Example: a user has many posts. * `GET /users/42/posts` (all posts for user 42) * `POST /users/42/posts` (create a post for user 42) ## Example Routes + Responses [Section titled “Example Routes + Responses”](#example-routes--responses) ### List users [Section titled “List users”](#list-users) `GET /users` ```json [ { "id": 1, "name": "Alice" }, { "id": 2, "name": "Bob" } ] ``` ### Get one user [Section titled “Get one user”](#get-one-user) `GET /users/1` ```json { "id": 1, "name": "Alice" } ``` ### Create a user [Section titled “Create a user”](#create-a-user) `POST /users` ```json { "id": 3, "name": "Charlie" } ``` ## Idempotency (Quick Concept) [Section titled “Idempotency (Quick Concept)”](#idempotency-quick-concept) Some methods are designed so repeating the same request has the same effect: * **GET** is idempotent (reading doesn’t change the server) * **PUT** is usually idempotent (replace with the same representation repeatedly) * **DELETE** is usually idempotent (deleting something twice keeps it deleted) * **POST** is usually **not** idempotent (posting twice often creates two resources) ## Status Codes [Section titled “Status Codes”](#status-codes) * **200 OK**: success * **201 Created**: successfully created * **400 Bad Request**: client sent wrong data * **401 Unauthorized**: you need to log in * **404 Not Found**: resource doesn’t exist * **500 Internal Server Error**: the server crashed or had a bug ## OpenAPI [Section titled “OpenAPI”](#openapi) Most REST APIs use an API spec for documentation and tooling. See: * [OpenAPI (Swagger)](/backend/api/styles/openapi) ## References [Section titled “References”](#references) * [RESTful API Design](https://restfulapi.net/) * [MDN HTTP Methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods) # SOAP **SOAP** is an older XML-based protocol used in many enterprise systems. It’s less common for new web apps, but still appears in long-lived integrations. ## Where SOAP Is Used [Section titled “Where SOAP Is Used”](#where-soap-is-used) * Legacy enterprise integrations (especially older vendor systems) * Regulated industries with long-running contracts (finance, government) ## Industry Examples (Companies) [Section titled “Industry Examples (Companies)”](#industry-examples-companies) * **PayPal**: historically offered SOAP APIs (alongside other options) * **Salesforce**: supports SOAP APIs for certain integrations > Note: company tech choices change over time; these are common, well-known examples. ## Tiny Example (Envelope) [Section titled “Tiny Example (Envelope)”](#tiny-example-envelope) ```xml 42 ``` ## References [Section titled “References”](#references) * [SOAP (Wikipedia)](https://en.wikipedia.org/wiki/SOAP) # Web Frameworks A **web framework** helps you build backend applications faster by giving you common building blocks and a consistent structure. Even though frameworks look different (Express, Django, Spring, Laravel, ASP.NET), they all solve the same problems. ## What Web Frameworks Have in Common [Section titled “What Web Frameworks Have in Common”](#what-web-frameworks-have-in-common) ### 1) Routing [Section titled “1) Routing”](#1-routing) **Routing** maps a request to a handler based on: * HTTP method (GET/POST/PUT/PATCH/DELETE) * URL path (like `/users/42`) Example idea (framework-agnostic): * `GET /users` → list users * `GET /users/:id` → get one user ### 2) Request + Response [Section titled “2) Request + Response”](#2-request--response) Frameworks provide an abstraction for: * the incoming request (headers, params, body) * the outgoing response (status code, JSON/HTML) You always do the same flow: 1. Read input from the request 2. Execute business logic 3. Return a response ### 3) Middleware / Filters / Interceptors [Section titled “3) Middleware / Filters / Interceptors”](#3-middleware--filters--interceptors) Most frameworks have a concept of “code that runs before/after your routes”. Typical uses: * logging * authentication * rate limiting * adding shared data to the request Naming differs: * Express: **middleware** * Spring: **filters**, **interceptors** * ASP.NET: **middleware** * NestJS: **guards**, **interceptors** ### 4) Controllers / Handlers [Section titled “4) Controllers / Handlers”](#4-controllers--handlers) A **handler** (sometimes called a controller action) is the function that runs when a route matches. Handlers are usually kept small and call services/helpers to do the real work. ### 5) Error Handling [Section titled “5) Error Handling”](#5-error-handling) Frameworks provide a standard way to: * handle “not found” routes * return consistent server errors * centralize error formatting ### 6) Parsing the Body [Section titled “6) Parsing the Body”](#6-parsing-the-body) Almost every backend needs to read request bodies: * JSON (`application/json`) * form data (`application/x-www-form-urlencoded`) * file uploads (`multipart/form-data`) Frameworks either include body parsing or plug it in. ### 7) Serving Static Files [Section titled “7) Serving Static Files”](#7-serving-static-files) Many frameworks can serve static files (images, CSS, downloads) or integrate with a reverse proxy. ### 8) Configuration + Environments [Section titled “8) Configuration + Environments”](#8-configuration--environments) Frameworks support: * configuration files * environment variables * dev vs production behavior ### 9) Database Integration (Common Pattern) [Section titled “9) Database Integration (Common Pattern)”](#9-database-integration-common-pattern) Frameworks don’t *have* to include databases, but most projects add: * an ORM/query builder * migrations * connection pooling ## One Concept, Many Frameworks (Mapping) [Section titled “One Concept, Many Frameworks (Mapping)”](#one-concept-many-frameworks-mapping) This is the same idea written in different frameworks: | Concept | Express (Node) | Django (Python) | Spring (Java) | Laravel (PHP) | | ------------- | ------------------------ | --------------------- | ----------------------- | ------------------------------ | | Route | `app.get('/users', ...)` | `path('users/', ...)` | `@GetMapping("/users")` | `Route::get('/users', ...)` | | Handler | function | view function/class | controller method | controller method | | Middleware | `app.use(...)` | middleware classes | filters/interceptors | middleware | | JSON response | `res.json(...)` | `JsonResponse(...)` | return object | `return response()->json(...)` | You don’t need to memorize syntax—focus on the concept and then learn the framework’s way of writing it. ## Choosing a Web Framework [Section titled “Choosing a Web Framework”](#choosing-a-web-framework) * **Minimal frameworks** (Express): more freedom, less structure * **Opinionated frameworks** (NestJS, Django, Spring): more structure, more built-in patterns ## Where Web Frameworks Are Used [Section titled “Where Web Frameworks Are Used”](#where-web-frameworks-are-used) * Backend APIs (REST, GraphQL) * Server-rendered apps (HTML pages) * Internal tools and admin panels Industry examples: * **Netflix**: large Node.js usage for web services * **Google**: heavy Java/Spring usage across many systems * **Shopify**: Ruby on Rails for large parts of its product ## Next [Section titled “Next”](#next) * Learn a concrete framework: [Express.js](/backend/api/express) * For API design concepts: [API Styles](/backend/api/styles) # Deployment & DevOps Deployment is how your code gets from your laptop to real users. DevOps is the set of practices and tooling that makes that process **reliable, repeatable, and observable**. ## Environments [Section titled “Environments”](#environments) Most projects use multiple environments: * **Development**: your machine * **Staging**: production-like testing * **Production**: real users **Example:** * Dev uses a local database. * Staging/Prod use managed Postgres. ## CI/CD (Continuous Integration / Continuous Deployment) [Section titled “CI/CD (Continuous Integration / Continuous Deployment)”](#cicd-continuous-integration--continuous-deployment) Automating the process of testing and releasing code. * **CI**: automatically run checks when you push code * **CD**: automatically deploy after CI succeeds (sometimes with manual approval) ### Example: Simple GitHub Actions Pipeline [Section titled “Example: Simple GitHub Actions Pipeline”](#example-simple-github-actions-pipeline) ```yaml name: CI on: pull_request: push: branches: [main] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: oven-sh/setup-bun@v2 - run: bun install - run: bun run build ``` ## Deployment Strategies (Common) [Section titled “Deployment Strategies (Common)”](#deployment-strategies-common) * **Rolling deploy**: replace instances gradually (common default) * **Blue/green**: two environments; switch traffic when ready * **Canary**: send a small percentage of users to the new version first ## Containerization (Docker) [Section titled “Containerization (Docker)”](#containerization-docker) **Docker** packages your app and everything it needs into a **container**. Why it’s useful: * same runtime everywhere (no “works on my machine”) * easy to run locally + in CI ### Example: Minimal Dockerfile for a Node/Express API [Section titled “Example: Minimal Dockerfile for a Node/Express API”](#example-minimal-dockerfile-for-a-nodeexpress-api) ```dockerfile FROM node:20-alpine WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci COPY . . EXPOSE 3000 CMD ["npm", "start"] ``` ### Example: Docker Compose (App + Database) [Section titled “Example: Docker Compose (App + Database)”](#example-docker-compose-app--database) ```yaml services: db: image: postgres:16 environment: POSTGRES_PASSWORD: postgres POSTGRES_DB: app ports: - '5432:5432' api: build: . environment: DATABASE_URL: postgresql://postgres:postgres@db:5432/app ports: - '3000:3000' depends_on: - db ``` ## Kubernetes (High Level) [Section titled “Kubernetes (High Level)”](#kubernetes-high-level) **Kubernetes** manages many containers: * restarts crashed containers * scales replicas up/down * exposes services It’s powerful, but adds complexity. Many teams start with a single VM or managed platform, then move to Kubernetes when needed. ## Version Control (Deployment Reality) [Section titled “Version Control (Deployment Reality)”](#version-control-deployment-reality) * **Git** (focus): commits, branches, merges * **GitHub** (focus): PRs, reviews, Actions Mentions: * **GitLab**: Git hosting + CI/CD * **Bitbucket**: Git hosting (often used with Jira) * **SVN**: older centralized version control system ## Operations (What Matters in Production) [Section titled “Operations (What Matters in Production)”](#operations-what-matters-in-production) As systems grow, these topics become important: * **Backups**: regular snapshots + restore testing * **Logging**: structured logs (JSON) * **Observability**: tracing/metrics/logs (e.g., OpenTelemetry) * **Monitoring / Error Tracking**: alerts and dashboards (e.g., Sentry) * **Profiling**: find performance bottlenecks * **Message brokers**: RabbitMQ, Kafka (often used for async workflows) * **Search engines**: Algolia, Elasticsearch ## Mini Case Studies [Section titled “Mini Case Studies”](#mini-case-studies) ### Case Study 1: From Laptop to Production (Small Team) [Section titled “Case Study 1: From Laptop to Production (Small Team)”](#case-study-1-from-laptop-to-production-small-team) 1. Developer opens a PR. 2. CI runs: lint/tests/build. 3. Merge to `main` triggers a deploy. 4. Deploy updates the running app (rolling) and health-checks it. **Typical “first production” setup:** * One VM * Nginx + your app process manager * Managed Postgres ### Case Study 2: Deploying with Database Migrations [Section titled “Case Study 2: Deploying with Database Migrations”](#case-study-2-deploying-with-database-migrations) Many backend deploys are really two deploys: * **Schema** change (migration) * **Code** change (new app version) Safe approach: 1. Deploy a migration that is backward-compatible (e.g., add a nullable column). 2. Deploy the new code that starts using it. 3. Later, clean up old columns/paths. This matters because during a rolling deploy, old and new versions can run at the same time. ### Case Study 3: Rollback When Something Breaks [Section titled “Case Study 3: Rollback When Something Breaks”](#case-study-3-rollback-when-something-breaks) If a release breaks production, a common rollback path is: * Roll back to the previous container/image/version. * Keep the database compatible (or have a plan for reversing migrations). **Rule of thumb:** reversible database migrations are hard; designing migrations to be backward-compatible is usually the safer strategy. ## References [Section titled “References”](#references) * [Docker Overview](https://docs.docker.com/get-started/overview/) * [GitHub Actions](https://docs.github.com/en/actions) # Architecture Patterns How you structure your backend so it stays understandable as it grows. Architecture patterns are trade-offs: you pick the shape that matches your team size, product complexity, and scaling needs. ## Monolith [Section titled “Monolith”](#monolith) The entire application (frontend, backend, database code) is in one project and deployed together. * **Pros**: Easy to start, easy to deploy initially. * **Cons**: Hard to scale. If one part breaks, everything breaks. **Example** * One repo, one backend service (`api`), one database. * Routes like `/users`, `/orders`, `/products` all live in the same codebase. Good for: early-stage products, small teams, and learning. ## Modular Monolith [Section titled “Modular Monolith”](#modular-monolith) A **modular monolith** is still one deployed application, but internally split into clear modules (often called “bounded contexts”): * `users/` module owns user logic * `billing/` module owns payments * `catalog/` module owns products **Why it matters:** it keeps the simplicity of one deployment, but reduces “spaghetti code”. **Example** * A single Express app, but you organize by feature modules and enforce module boundaries. ## Layered Architecture (Common) [Section titled “Layered Architecture (Common)”](#layered-architecture-common) A very common structure is “layers”: * **Controllers / Routes**: HTTP details (URLs, status codes) * **Services**: business logic (rules) * **Data Access / Repositories**: database queries **Example** * `POST /orders` -> controller validates basic shape -> service checks inventory -> repository writes to DB. ## Microservices [Section titled “Microservices”](#microservices) Breaking the app into many small services (e.g., User Service, Payment Service) that talk to each other. * **Pros**: Scale parts independently. Different teams can work on different services. * **Cons**: Very complex to manage. **What becomes harder:** * Service-to-service auth * Debugging (distributed logs/tracing) * Data consistency across services * Local development (many things to run) **Example** * `users-service` owns the Users DB. * `orders-service` owns the Orders DB. * They communicate via HTTP/gRPC or a message broker. ## Serverless [Section titled “Serverless”](#serverless) You write simple functions (e.g., AWS Lambda) and the cloud provider runs them only when needed. You don’t manage servers. **Typical uses** * Webhooks * Background jobs * “Glue code” integrations **Example** * A function `onPaymentSucceeded()` updates an order status after Stripe webhook events. ## Service-Oriented Architecture (SOA) [Section titled “Service-Oriented Architecture (SOA)”](#service-oriented-architecture-soa) **SOA** is an older (but still relevant) approach where an application is built from reusable services that communicate over a network (often via an Enterprise Service Bus / shared integration layer). * **Compared to microservices**: SOA services are often larger and share more infrastructure; microservices tend to be smaller and independently deployable. ## Event-Driven Architecture (EDA) [Section titled “Event-Driven Architecture (EDA)”](#event-driven-architecture-eda) In an **event-driven** system, services communicate by publishing events like “OrderCreated” or “UserRegistered”. Other services subscribe and react. * Often implemented via a **message broker** (RabbitMQ, Kafka). **Example** * `orders-service` publishes `OrderCreated`. * `email-service` listens and sends a confirmation email. This reduces direct dependencies between services, but introduces “eventual consistency”. ## Quick Guidance (Picking a Pattern) [Section titled “Quick Guidance (Picking a Pattern)”](#quick-guidance-picking-a-pattern) * **Just starting**: monolith or modular monolith. * **Multiple teams + independent scaling**: microservices. * **Many integrations / async workflows**: event-driven architecture. * **Spiky traffic + small functions**: serverless. ## Mini Case Studies [Section titled “Mini Case Studies”](#mini-case-studies) ### Case Study 1: E-Commerce Growth Path [Section titled “Case Study 1: E-Commerce Growth Path”](#case-study-1-e-commerce-growth-path) 1. **Monolith (start)**: one backend + one database. 2. **Modular monolith (next)**: split into modules: `users`, `catalog`, `orders`, `billing`. 3. **Microservices (later)**: extract `billing` and `notifications` first (they often have very different scaling and reliability needs). **Why this order works:** you keep deployment simple early, and only pay the microservices complexity cost when multiple teams and scaling pressure justify it. ### Case Study 2: Order Processing with Events [Section titled “Case Study 2: Order Processing with Events”](#case-study-2-order-processing-with-events) You want to avoid a big synchronous chain like “orders calls payments calls email calls shipping”. * `orders-service` stores the order and publishes `OrderCreated`. * `payments-service` listens, charges the card, publishes `PaymentSucceeded` or `PaymentFailed`. * `email-service` listens and sends confirmation. **Trade-off:** the UI might show “processing” for a short time because systems become eventually consistent. ## References [Section titled “References”](#references) * [Microservices Architecture](https://microservices.io/) # Web Servers & Reverse Proxies A **web server** (or reverse proxy) often sits in front of your backend application. Even if your backend is an Express app, production setups commonly look like:  ## What a Reverse Proxy Does [Section titled “What a Reverse Proxy Does”](#what-a-reverse-proxy-does) A reverse proxy receives requests from clients and forwards them to your application. Common jobs: * **TLS termination**: handle HTTPS certificates (so your app can run plain HTTP internally) * **Static files**: serve assets efficiently * **Compression**: gzip/brotli responses * **Caching**: cache responses for speed * **Load balancing**: distribute traffic across multiple app instances * **Rate limiting**: protect against abuse * **Request limits**: cap upload sizes / header sizes ## Popular Choices [Section titled “Popular Choices”](#popular-choices) * **Nginx**: very popular, fast, flexible * **Apache**: older but feature-rich * **Caddy**: great developer experience; HTTPS is often very easy * **Traefik**: common in container/Kubernetes environments ## Express Behind a Reverse Proxy (Key Notes) [Section titled “Express Behind a Reverse Proxy (Key Notes)”](#express-behind-a-reverse-proxy-key-notes) * If your reverse proxy is doing TLS, your Express app may only see internal HTTP unless you configure it correctly. * If you use cookies with `secure: true`, you usually need `app.set("trust proxy", 1)` so Express understands it’s really HTTPS. ## Example: Nginx Reverse Proxy to Express [Section titled “Example: Nginx Reverse Proxy to Express”](#example-nginx-reverse-proxy-to-express) This example sends all traffic to an Express app running on `localhost:3000`. ```nginx server { listen 80; server_name example.com; location / { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } ``` ## WebSockets, SSE, and Streaming Through a Proxy [Section titled “WebSockets, SSE, and Streaming Through a Proxy”](#websockets-sse-and-streaming-through-a-proxy) Real-time features often require proxy settings to avoid buffering and to support connection upgrades. ### WebSockets [Section titled “WebSockets”](#websockets) WebSockets require an HTTP upgrade. ```nginx location /ws { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host; } ``` ### SSE / HTTP Streaming [Section titled “SSE / HTTP Streaming”](#sse--http-streaming) SSE and streaming rely on long-lived connections and sending chunks quickly. ```nginx location /events { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_buffering off; proxy_cache off; } ``` ## Load Balancing (Concept) [Section titled “Load Balancing (Concept)”](#load-balancing-concept) When your app runs multiple instances:  **Important:** real-time connections (WebSockets/SSE) can be sensitive to load balancing. You may need sticky sessions or a shared pub/sub layer depending on your design. ## Mini Case Studies [Section titled “Mini Case Studies”](#mini-case-studies) ### Case Study 1: Single VM Deployment (Common First Step) [Section titled “Case Study 1: Single VM Deployment (Common First Step)”](#case-study-1-single-vm-deployment-common-first-step) * A single Linux VM runs: * **Nginx** on `:443` (HTTPS) * **Express** on `:3000` * Nginx terminates TLS and forwards requests to Express. * Logs: * Nginx access/error logs * App logs (stdout) collected by your process manager This setup is cheap, understandable, and handles a lot of traffic for small/medium apps. ### Case Study 2: Real-Time Features Behind a Proxy [Section titled “Case Study 2: Real-Time Features Behind a Proxy”](#case-study-2-real-time-features-behind-a-proxy) * You add **SSE** at `/events` and/or **WebSockets** at `/ws`. * You must ensure: * WebSockets: `Upgrade` headers are forwarded. * SSE/streaming: proxy buffering is off, so events arrive immediately. If you later add multiple app instances behind a load balancer, you may need: * **Sticky sessions** (same client goes to the same instance), or * A shared pub/sub (e.g., Redis pub/sub) so updates reach clients no matter which instance they’re connected to. # Databases A **Database** is an organized collection of data, so that it can be easily accessed and managed. ## SQL (Relational Databases) [Section titled “SQL (Relational Databases)”](#sql-relational-databases) SQL (Structured Query Language) databases store data in **tables** (like Excel sheets) with rows and columns. They are good for structured data and complex queries. ### PostgreSQL [Section titled “PostgreSQL”](#postgresql) **PostgreSQL** (or Postgres) is an advanced, open-source object-relational database system. * **Reliable**: Known for data integrity and correctness. * **Features**: Supports JSON, complex queries, and custom types. * **ACID Compliant**: Ensures transactions are processed reliably (Atomicity, Consistency, Isolation, Durability). For a deeper, practical guide with raw SQL examples (constraints, indexes, joins, transactions, JSONB), see: * [PostgreSQL (Postgres)](/backend/data/postgresql)  ### Other SQL Databases [Section titled “Other SQL Databases”](#other-sql-databases) * **MySQL**: Very popular, powers WordPress. * **MariaDB**: A popular MySQL-compatible fork. * **SQLite**: A file-based database. Good for small apps or mobile. * **Oracle**: Enterprise database (often used in large organizations). * **Microsoft SQL Server**: Enterprise database (common in Windows/.NET environments). ## NoSQL (Non-Relational) [Section titled “NoSQL (Non-Relational)”](#nosql-non-relational) NoSQL databases do *not* use tables. They might use documents (JSON), key-value pairs, or graphs. * **MongoDb**: Stores data like JSON documents. Flexible schema. * **Redis**: In-memory key-value store. Very fast. Often used for **Caching**. ## Concepts [Section titled “Concepts”](#concepts) * **Normalization**: Organizing data to reduce redundancy (duplicates). * **Transactions**: A group of operations that either *all* succeed or *all* fail. * **Indexing**: A way to speed up searching in a database (like the index at the back of a book). ### Replication (Mention) [Section titled “Replication (Mention)”](#replication-mention) **Replication** means keeping copies of your data across multiple machines. * Improves **availability** (if one node goes down, another can serve reads). * Can improve **read performance**. ### Sharding (Mention) [Section titled “Sharding (Mention)”](#sharding-mention) **Sharding** splits a large dataset into smaller parts (shards) across multiple databases/servers. * Improves **horizontal scalability**. * Adds complexity (routing queries, rebalancing shards). ## References [Section titled “References”](#references) * [PostgreSQL Official Site](https://www.postgresql.org/) * [SQL vs NoSQL](https://www.mongodb.com/nosql-explained/nosql-vs-sql) # PostgreSQL (Postgres) PostgreSQL (Postgres) is a popular SQL database used in many production web backends. It’s often chosen because it’s: * **Correct and reliable** (strong consistency + transactions) * **Flexible** (powerful SQL + `jsonb` + extensions) * **Scalable** (good indexing, replication options, mature tooling) ## Tables, Relations, and Foreign Keys [Section titled “Tables, Relations, and Foreign Keys”](#tables-relations-and-foreign-keys) Relational databases model data as tables and relationships. Key idea: **foreign keys** connect rows across tables and protect integrity. * A `posts.user_id` value must match an existing `users.id`. * If you delete a user, you decide what happens to their posts (block deletion, cascade delete, set null, etc.).  ## Constraints (Data Integrity) [Section titled “Constraints (Data Integrity)”](#constraints-data-integrity) Constraints prevent invalid data from entering your database. Common examples: * `PRIMARY KEY`: unique identifier * `UNIQUE`: prevent duplicates (e.g., email) * `NOT NULL`: required field * `CHECK`: enforce a rule (e.g., price >= 0) ```sql CREATE TABLE users ( id SERIAL PRIMARY KEY, email TEXT NOT NULL UNIQUE, name TEXT NOT NULL, created_at TIMESTAMP NOT NULL DEFAULT NOW() ); CREATE TABLE posts ( id SERIAL PRIMARY KEY, user_id INT NOT NULL REFERENCES users(id) ON DELETE CASCADE, title TEXT NOT NULL, body TEXT NOT NULL ); ``` ## Indexes (Performance) [Section titled “Indexes (Performance)”](#indexes-performance) Indexes speed up lookups (at the cost of storage and slower writes). Typical indexes: * `users.email` for login / lookup * `posts.user_id` for “all posts by a user” ```sql CREATE INDEX idx_posts_user_id ON posts(user_id); ``` ## Joins (Querying Across Tables) [Section titled “Joins (Querying Across Tables)”](#joins-querying-across-tables) Joins let you combine related rows. Example: get posts with author name ```sql SELECT p.id, p.title, u.name AS author FROM posts p JOIN users u ON u.id = p.user_id ORDER BY p.id DESC LIMIT 20; ``` ## Transactions (Correctness) [Section titled “Transactions (Correctness)”](#transactions-correctness) A **transaction** groups multiple operations so they succeed or fail together. Typical use cases: * Create an order + order items * Transfer money between accounts * Decrement inventory when placing an order Postgres transactions also support different **isolation levels** (how strongly concurrent requests are separated). You’ll often start with the default and go deeper only when you hit concurrency bugs. ## JSONB (Relational + Document) [Section titled “JSONB (Relational + Document)”](#jsonb-relational--document) Postgres supports `jsonb`, which is useful when you need flexible fields while still keeping relational structure. Example uses: * store provider payloads (webhooks) * store feature flags / preferences * store “metadata” fields that evolves over time ## References [Section titled “References”](#references) * [PostgreSQL Official Site](https://www.postgresql.org/) # Prisma ORM Prisma is an ORM for Node.js/TypeScript that helps you work with a SQL database (often Postgres) using generated, typed queries. ## What Prisma Gives You [Section titled “What Prisma Gives You”](#what-prisma-gives-you) * A **schema** file (`schema.prisma`) where you define models * **Migrations** generated from schema changes * A generated **Prisma Client** you use in code ## Example: Prisma Schema (Postgres) [Section titled “Example: Prisma Schema (Postgres)”](#example-prisma-schema-postgres) schema.prisma ```prisma datasource db { provider = "postgresql" url = env("DATABASE_URL") } generator client { provider = "prisma-client-js" } model User { id Int @id @default(autoincrement()) email String @unique name String createdAt DateTime @default(now()) posts Post[] } model Post { id Int @id @default(autoincrement()) title String body String userId Int user User @relation(fields: [userId], references: [id], onDelete: Cascade) } ``` ## Example: Basic Prisma Queries [Section titled “Example: Basic Prisma Queries”](#example-basic-prisma-queries) ```ts const users = await prisma.user.findMany(); const user = await prisma.user.findUnique({ where: { email: 'a@b.com' } }); const posts = await prisma.post.findMany({ where: { userId: 123 }, orderBy: { id: 'desc' }, take: 10, }); ``` ## Raw SQL vs Prisma (Same Idea, Two Approaches) [Section titled “Raw SQL vs Prisma (Same Idea, Two Approaches)”](#raw-sql-vs-prisma-same-idea-two-approaches) Even if you use Prisma, you should be able to read SQL. ### Example: “Get the latest 20 posts with author name” [Section titled “Example: “Get the latest 20 posts with author name””](#example-get-the-latest-20-posts-with-author-name) Raw SQL: ```sql SELECT p.id, p.title, u.name AS author FROM posts p JOIN users u ON u.id = p.user_id ORDER BY p.id DESC LIMIT 20; ``` Prisma: ```ts const posts = await prisma.post.findMany({ orderBy: { id: 'desc' }, take: 20, include: { user: { select: { name: true } } }, }); ``` ### When You Still Use Raw SQL [Section titled “When You Still Use Raw SQL”](#when-you-still-use-raw-sql) * Complex reporting queries * Vendor-specific features (Postgres `jsonb`, full-text search, advanced aggregations) * Performance tuning (custom SQL + careful indexing) Prisma supports parameterized raw SQL (safe against SQL injection): ```ts const userId = 123; const posts = await prisma.$queryRaw` SELECT id, title FROM posts WHERE user_id = ${userId} ORDER BY id DESC LIMIT 10 `; ``` ## Express Example: Prisma vs Raw SQL [Section titled “Express Example: Prisma vs Raw SQL”](#express-example-prisma-vs-raw-sql) Two endpoints returning the same kind of data: * one uses **Prisma ORM queries** * one uses a **raw SQL query** ### Setup [Section titled “Setup”](#setup) ```ts import express from 'express'; import { PrismaClient } from '@prisma/client'; const app = express(); const prisma = new PrismaClient(); app.use(express.json()); ``` ### Endpoint 1: Prisma ORM Query [Section titled “Endpoint 1: Prisma ORM Query”](#endpoint-1-prisma-orm-query) ```ts app.get('/api/users/:id/posts', async (req, res) => { const userId = Number(req.params.id); if (Number.isNaN(userId)) return res.status(400).json({ error: 'Invalid user id' }); const posts = await prisma.post.findMany({ where: { userId }, orderBy: { id: 'desc' }, take: 10, select: { id: true, title: true }, }); res.json({ posts }); }); ``` ### Endpoint 2: Raw SQL Query (Same Result) [Section titled “Endpoint 2: Raw SQL Query (Same Result)”](#endpoint-2-raw-sql-query-same-result) ```ts app.get('/api/users/:id/posts-raw', async (req, res) => { const userId = Number(req.params.id); if (Number.isNaN(userId)) return res.status(400).json({ error: 'Invalid user id' }); const rows = await prisma.$queryRaw` SELECT id, title FROM posts WHERE user_id = ${userId} ORDER BY id DESC LIMIT 10 `; res.json({ posts: rows }); }); ``` ## Prisma Migrations (Typical Workflow) [Section titled “Prisma Migrations (Typical Workflow)”](#prisma-migrations-typical-workflow) * Edit `schema.prisma` * Run a migration to update your database schema * Prisma generates migration files and updates the client ## References [Section titled “References”](#references) * [Prisma ORM](https://www.prisma.io/) # Data Management Tools Managing raw SQL can be hard. We use tools to help us. ## ORM (Object-Relational Mapping) [Section titled “ORM (Object-Relational Mapping)”](#orm-object-relational-mapping) An **ORM** lets you interact with your database using your programming language (like JavaScript) instead of writing raw SQL queries. * **Prisma**: Modern ORM for TypeScript and Node.js. Very popular. * **Drizzle**: Lightweight and fast ORM. ### Example (Prisma Query) [Section titled “Example (Prisma Query)”](#example-prisma-query) Instead of `SELECT * FROM users`, you write: ```typescript const users = await prisma.user.findMany(); ``` For a deeper guide (schema, queries, raw SQL, and an Express example), see: * [Prisma ORM](/backend/data/prisma) ## Migrations [Section titled “Migrations”](#migrations) **Database Migrations** are like version control for your database schema. They treat changes to your database structure (like adding a column) as code. * Helps teams stay in sync. * Allows you to roll back changes if something breaks. ### Prisma Migrations (Typical Workflow) [Section titled “Prisma Migrations (Typical Workflow)”](#prisma-migrations-typical-workflow) * Edit `schema.prisma` * Run a migration to update your database schema * Prisma generates migration files and updates the client ## Caching [Section titled “Caching”](#caching) Caching stores copies of data in a faster storage (usually RAM) so future requests for that data can be served faster. * **Browser Caching**: The browser saves images/css. * **Server Caching**: The server saves HTML or API responses. * **Database Caching**: Using **Redis** to store frequent queries. * **Memcached** (mentioned): A simple, in-memory cache (key/value). Often used for very fast caching, but with fewer features than Redis.  ## References [Section titled “References”](#references) * [Prisma ORM](https://www.prisma.io/) * [Redis Caching](https://redis.io/) # Backend Languages & Runtimes ## Languages [Section titled “Languages”](#languages) Backend development can be done in many programming languages. Here are some of the most popular ones: * **JavaScript**: Originally for frontend, but with Node.js, it’s very popular for backend too. * **PHP**: One of the oldest web languages. Used by WordPress. * **Java**: Strong, typed language. Very popular in big companies (enterprise). * **Python**: Easy to learn, great for data science and AI backends. * **C#**: Used with Microsoft’s .NET framework. * **Go (Golang)**: Created by Google. Very fast and good for high performance. * **Rust**: Focuses on safety and speed. Harder to learn but very efficient. * **Ruby**: Known for the “Ruby on Rails” framework which helps build apps quickly. ## Runtimes [Section titled “Runtimes”](#runtimes) A **Runtime** is the environment where your code runs. ### Node.js [Section titled “Node.js”](#nodejs) > Focus Topic **Node.js** is a runtime environment that lets you run JavaScript outside the browser (on the server). It is built on Chrome’s V8 JavaScript engine. * **Non-blocking I/O**: It doesn’t wait for tasks (like reading a file) to finish before moving to the next one. This makes it very fast for handling many connections at once. * **Event-Driven**: It uses events (like a user click or a data arrival) to trigger code. * **Single-Threaded**: It uses one main thread to handle requests, but delegates heavy work to the system so it doesn’t get stuck.  ### Other Runtimes [Section titled “Other Runtimes”](#other-runtimes) * **Bun**: A newer, faster runtime that aims to replace Node.js. It includes a bundler and test runner. * **Deno**: Created by the same person who made Node.js, to fix some of its design issues. ## References [Section titled “References”](#references) * [Node.js Official Site](https://nodejs.org/en/) * [Bun](https://bun.sh/) * [Deno](https://deno.com/) # Package Managers A **Package Manager** is a tool that automates the process of installing, upgrading, configuring, and removing software packages (libraries) in your project. ## npm (Node Package Manager) [Section titled “npm (Node Package Manager)”](#npm-node-package-manager) > Focus Topic **npm** is the default package manager for Node.js. It is the world’s largest software registry. When you install Node.js, you get npm automatically. ### Key Commands [Section titled “Key Commands”](#key-commands) * `npm init`: Sets up a new project (creates `package.json`). * `npm install `: Downloads a library and adds it to your project. * `npm start`: Runs the start script defined in your package file. ### structure [Section titled “structure”](#structure)  * **package.json**: The manifest file. It lists your project’s name, version, and dependencies. * **node\_modules**: The folder where the actual code of the downloaded packages lives. Do not edit files here! * **package-lock.json**: Ensures everyone on your team installs the *exact* same versions of packages. ## Other Package Managers [Section titled “Other Package Managers”](#other-package-managers) * **yarn**: Created by Facebook to be faster and more secure than npm (though npm has caught up). * **pnpm**: “Performant npm”. It saves disk space by using a shared store for packages, so you don’t have duplicates on your computer. ## References [Section titled “References”](#references) * [npm Docs](https://docs.npmjs.com/) * [pnpm](https://pnpm.io/) # Backend Development Backend Development refers to the server-side software that powers websites and applications. It handles the “business logic”, database interactions, and API integrations that users don’t see directly. This module covers the essential concepts, tools, and practices for modern backend engineering. ## Learning Path [Section titled “Learning Path”](#learning-path) We have divided the learning material into the following sections: ### 1. Foundations [Section titled “1. Foundations”](#1-foundations) Start here to understand the languages and runtimes that power the backend. * [Languages & Runtimes](/backend/foundations/languages): Node.js, Python, Go. * [Package Managers](/backend/foundations/package-managers): npm, yarn, pnpm. ### 2. API Development [Section titled “2. API Development”](#2-api-development) Learn how to build interfaces for your application. * [API Styles](/backend/api/styles): REST (Focus), GraphQL. * [Web Frameworks](/backend/api/web-frameworks): Common concepts you can use anywhere. * [Express.js](/backend/api/express): Routing, middleware, routers. * [Real-Time Data](/backend/api/real-time): Long Polling, SSE, WebSockets. ### 3. Data & Persistence [Section titled “3. Data & Persistence”](#3-data--persistence) How to store and manage data efficiently. * [Databases](/backend/data/databases): PostgreSQL (Focus), NoSQL. * [PostgreSQL (Postgres)](/backend/data/postgresql): Tables, constraints, indexes, joins. * [Data Tools](/backend/data/tools): ORMs, migrations, caching. * [Prisma ORM](/backend/data/prisma): Schema, queries, raw SQL vs ORM, Express example. ### 4. Security [Section titled “4. Security”](#4-security) Protecting your application and users. * [Web Security](/backend/security/web-security): Encryption, Hashing, OWASP. * [Authentication](/backend/security/auth): JWT, OAuth, RBAC. ### 5. Architecture & Deployment [Section titled “5. Architecture & Deployment”](#5-architecture--deployment) Scaling and shipping your code. * [Architecture Patterns](/backend/architecture/patterns): Monolith, microservices, serverless. * [Web Servers & Reverse Proxies](/backend/architecture/web-servers): Nginx, TLS termination, load balancing. * [Deployment & DevOps](/backend/architecture/deployment-devops): CI/CD, Docker, operations. ## Roadmap Reference [Section titled “Roadmap Reference”](#roadmap-reference) This curriculum follows industry standards for backend development. * [Backend Developer Roadmap](https://roadmap.sh/backend) # Auth & Authorization **Authentication** is knowing *who* you are. **Authorization** is knowing *what* you are allowed to do. ## Authentication (AuthN) [Section titled “Authentication (AuthN)”](#authentication-authn) How do we log users in? ### Basic Auth [Section titled “Basic Auth”](#basic-auth) Sending the username and password with every single request. Not very secure unless used with HTTPS. ### Session/Cookie Auth [Section titled “Session/Cookie Auth”](#sessioncookie-auth) 1. User logs in. 2. Server creates a “session” ID and stores it. 3. Server sends the ID to the browser in a **Cookie**. 4. Browser automatically sends the Cookie on next visits. ### Token-Based (JWT) [Section titled “Token-Based (JWT)”](#token-based-jwt) **JSON Web Token** is a standard (RFC 7519) for representing claims securely. 1. User logs in. 2. Server creates a signed JWT (a long string) containing user info. 3. Server sends it to Client. 4. Client stores it (e.g., Local Storage) and sends it in the Header (`Authorization: Bearer `). **Pros**: Stateless (Server doesn’t need to store sessions). Good for mobile and microservices.  ### OAuth [Section titled “OAuth”](#oauth) Using other services to log in (e.g., “Log in with Google”). **Mentions**: * **OpenID Connect (OIDC)**: Built on OAuth 2.0, adds a standard identity layer (who the user is). * **SAML**: Often used in enterprise Single Sign-On (SSO) for internal tools. ## Authorization (AuthZ) [Section titled “Authorization (AuthZ)”](#authorization-authz) Once logged in, what can they do? * **RBAC (Role-Based Access Control)**: Assign roles (Admin, Editor, User). Permissions are attached to the Role. * **ABAC (Attribute-Based)**: More complex rules (e.g., “Can edit if document belongs to user AND it is a weekday”). * **ACL (Access Control List)**: Permissions are stored per resource (e.g., “this document can be read by Alice and Bob”). ## References [Section titled “References”](#references) * [JWT.io](https://jwt.io/) * [OAuth 2.0 Explained](https://oauth.net/2/) # Web Security Security is crucial in backend development to protect user data and preventing attacks. ## Core Concepts [Section titled “Core Concepts”](#core-concepts) ### Hashing [Section titled “Hashing”](#hashing) Hashing turns data (like a password) into a random string of characters. You cannot turn the hash back into the password. * **Used for**: Storing passwords securely. * **Used for**: Storing passwords securely. * **Algorithms**: * **MD5**: Old and broken for security purposes (avoid for passwords). * **SHA (SHA-1 / SHA-2 / SHA-3)**: A family of hashing algorithms. For security, prefer modern variants like SHA-256/512. * **bcrypt / scrypt / Argon2**: Best practice for password hashing (slow by design). ### Encryption [Section titled “Encryption”](#encryption) Encryption turns data into code, but you *can* reverse it if you have the “Key”. * **HTTPS**: Encrypts traffic between the browser and server. * **HTTPS**: Encrypts traffic between the browser and server. * **TLS/SSL**: The protocols behind HTTPS. * **RSA**: An asymmetric (public/private key) algorithm historically used in TLS and for signing/encryption (less common for key exchange today). * **Diffie-Hellman**: A key-exchange method used to agree on a shared secret over an untrusted network (modern TLS often uses ECDHE, an elliptic-curve variant). ### CORS (Cross-Origin Resource Sharing) [Section titled “CORS (Cross-Origin Resource Sharing)”](#cors-cross-origin-resource-sharing) A browser feature that stops a website on domain A (evil.com) from stealing data from domain B (bank.com). You must explicitly allow domains in your backend. ### CSP (Content Security Policy) [Section titled “CSP (Content Security Policy)”](#csp-content-security-policy) **CSP** helps reduce attacks like **XSS** by telling the browser which sources are allowed to load scripts, styles, images, etc. Example header: `Content-Security-Policy: default-src 'self'; script-src 'self'` ## Common Vulnerabilities (OWASP) [Section titled “Common Vulnerabilities (OWASP)”](#common-vulnerabilities-owasp) **OWASP** is an organization that lists the top security risks. Common ones: 1. **Injection**: Sending malicious SQL commands (SQL Injection). 2. **Broken Authentication**: Letting attackers log in as someone else. ## References [Section titled “References”](#references) * [OWASP Top 10](https://owasp.org/www-project-top-ten/) * [MDN CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) * [MDN Content Security Policy (CSP)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP)