| [ef1c1c7] | 1 | # Advanced Database Development
|
|---|
| 2 |
|
|---|
| 3 | EduBerza's users trade virtual money and virtual crypto with each other and with a simulated
|
|---|
| 4 | market. Up to P6 the prototype only knew market orders that filled immediately against the
|
|---|
| 5 | latest price, so an order was either untouched or completely done. This phase adds what makes an
|
|---|
| 6 | exchange consistent once orders can **wait in an order book, fill in parts, and trade with each
|
|---|
| 7 | other**, and puts every rule that keeps orders, trades and balances in agreement into the
|
|---|
| 8 | database itself — so it holds no matter who writes the data (the CLI, the market bot, a script,
|
|---|
| 9 | or someone typing SQL in DBeaver).
|
|---|
| 10 |
|
|---|
| 11 | Only rules that span several rows or several tables are listed. `NOT NULL`, `UNIQUE`, `CHECK`,
|
|---|
| 12 | primary and foreign keys are P2 ([RelationalDesign](../P2-RelationalDesign/RelationalDesign.md))
|
|---|
| 13 | and are not presented as P7 features.
|
|---|
| 14 |
|
|---|
| 15 | All of it is in [`server/db/advanced_db.sql`](../../server/db/advanced_db.sql), run by `-init`
|
|---|
| 16 | between `schema_creation.sql` and `data_load.sql`. Every rule is exercised by
|
|---|
| 17 | [`server/db/advanced_db_tests.sql`](../../server/db/advanced_db_tests.sql) (see
|
|---|
| 18 | [Tests](#tests-proving-the-rules)).
|
|---|
| 19 |
|
|---|
| 20 | ## Overview
|
|---|
| 21 |
|
|---|
| 22 | | # | Requirement | Triggers | Procedures / functions | Views | Tables affected |
|
|---|
| 23 | |---|---|---|---|---|---|
|
|---|
| 24 | | 1 | [Order lifecycle and filled/remaining consistency](#order-lifecycle-and-filledremaining-consistency) | `orders_lifecycle` | | | `orders` |
|
|---|
| 25 | | 2 | [Trade consistency](#trade-consistency) | `market_trades_validate`, `market_trades_fill`, `market_trades_immutable` | `execute_trade` | | `market_trades`, `orders`, `users`, `holdings`, `transactions` |
|
|---|
| 26 | | 3 | [Balance and reservation consistency](#balance-and-reservation-consistency) | `reserved_cash_matches_orders`, `reserved_crypto_matches_orders`, `cash_matches_ledger` (deferred) | `order_reservation` | | `users`, `holdings`, `orders`, `transactions` |
|
|---|
| 27 | | 4 | [Placing and cancelling orders](#placing-and-cancelling-orders) | | `place_order`, `match_order`, `cancel_order` | | all of the above |
|
|---|
| 28 | | 5 | [Automatic recording of order events](#automatic-recording-of-order-events) | `orders_events` | | | `order_events` (new) |
|
|---|
| 29 | | 6 | [Views for derived trading data](#views-for-derived-trading-data) | | | `v_order_book`, `v_active_orders`, `v_order_history`, `v_trader_balances` | read-only |
|
|---|
| 30 | | 7 | [Background job: filling resting limit orders](#background-job-filling-resting-limit-orders) | | `fill_marketable_orders` | | `orders`, `market_trades`, … |
|
|---|
| 31 |
|
|---|
| 32 | ### Schema additions
|
|---|
| 33 |
|
|---|
| 34 | The existing design had no place for three things the requirements need, so four columns were
|
|---|
| 35 | added — nothing else in the design changed:
|
|---|
| 36 |
|
|---|
| 37 | | Column | Why it is needed |
|
|---|
| 38 | |---|---|
|
|---|
| 39 | | `orders.filled_quantity` (+ status value `partially_filled`) | "filled / remaining quantity" cannot be kept consistent without storing how much was filled; remaining = `quantity − filled_quantity` |
|
|---|
| 40 | | `users.reserved_balance` | cash committed to open buy orders must be set aside somewhere; `holdings.reserved_quantity` already did this for crypto on sell orders |
|
|---|
| 41 | | `market_trades.buy_order_id`, `market_trades.sell_order_id` | "a trade can only happen between compatible orders" needs the trade to say which orders it filled. `NULL` on a side means the simulated market was the counterparty; the bot's price ticks have both `NULL` |
|
|---|
| 42 |
|
|---|
| 43 | One new table, `order_events`, holds the automatically recorded events (requirement 5).
|
|---|
| 44 |
|
|---|
| 45 | ## Order lifecycle and filled/remaining consistency
|
|---|
| 46 |
|
|---|
| 47 | ### Data requirements description
|
|---|
| 48 |
|
|---|
| 49 | **Business rule.**
|
|---|
| 50 |
|
|---|
| 51 | - An order's status follows from how much of it has been filled: nothing filled → `open`,
|
|---|
| 52 | partly filled → `partially_filled`, completely filled → `executed`. The only status that is
|
|---|
| 53 | set explicitly is `cancelled`, and only on an order that is still active.
|
|---|
| 54 | - `executed` and `cancelled` are final — such an order can never be processed again (no further
|
|---|
| 55 | fills, no cancellation, no changes).
|
|---|
| 56 | - `filled_quantity` only grows, never exceeds `quantity`, and only changes when a trade fills
|
|---|
| 57 | the order.
|
|---|
| 58 | - What was ordered — user, market, side, type, quantity, price, time of placement — never
|
|---|
| 59 | changes after placement.
|
|---|
| 60 | - `executed_at` is set exactly when the order becomes `executed`.
|
|---|
| 61 | - New orders are only accepted on active markets, need a positive price, and start unfilled
|
|---|
| 62 | (the one exception is importing an order that was completely executed in the past, used by the
|
|---|
| 63 | sample data).
|
|---|
| 64 |
|
|---|
| 65 | **Why it is non-trivial.** The rule compares the *old* and the *new* version of a row (valid
|
|---|
| 66 | transitions, "final" states, "only grows", immutable columns) and ties two columns together
|
|---|
| 67 | (status ↔ filled quantity). A `CHECK` constraint only sees one version of one row. It also
|
|---|
| 68 | depends on *who* changes `filled_quantity` — a trade may, a manual `UPDATE` may not.
|
|---|
| 69 |
|
|---|
| 70 | **PostgreSQL feature.** `BEFORE INSERT OR UPDATE` row trigger on `orders`. The trade trigger
|
|---|
| 71 | (requirement 2) sets a transaction-local setting (`set_config('eduberza.trade_fill', 'on', true)`)
|
|---|
| 72 | while it fills an order; the lifecycle trigger only accepts a change of `filled_quantity` when
|
|---|
| 73 | that setting is on.
|
|---|
| 74 |
|
|---|
| 75 | **Tables affected.** `orders` (reads `markets` for the active check).
|
|---|
| 76 |
|
|---|
| 77 | ### Implementation
|
|---|
| 78 |
|
|---|
| 79 | #### Triggers
|
|---|
| 80 |
|
|---|
| 81 | ```sql
|
|---|
| 82 | CREATE OR REPLACE FUNCTION project.trg_orders_lifecycle()
|
|---|
| 83 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 84 | DECLARE
|
|---|
| 85 | v_derived varchar(20);
|
|---|
| 86 | BEGIN
|
|---|
| 87 | IF TG_OP = 'UPDATE' THEN
|
|---|
| 88 | IF OLD.status IN ('executed', 'cancelled') THEN
|
|---|
| 89 | RAISE EXCEPTION 'order % is already % and cannot be processed again', OLD.id, OLD.status
|
|---|
| 90 | USING ERRCODE = 'check_violation';
|
|---|
| 91 | END IF;
|
|---|
| 92 | IF (NEW.user_id, NEW.market_id, NEW.side, NEW.type, NEW.quantity, NEW.price, NEW.placed_at)
|
|---|
| 93 | IS DISTINCT FROM
|
|---|
| 94 | (OLD.user_id, OLD.market_id, OLD.side, OLD.type, OLD.quantity, OLD.price, OLD.placed_at) THEN
|
|---|
| 95 | RAISE EXCEPTION 'user, market, side, type, quantity, price and placed_at of an order cannot change'
|
|---|
| 96 | USING ERRCODE = 'check_violation';
|
|---|
| 97 | END IF;
|
|---|
| 98 | IF NEW.filled_quantity <> OLD.filled_quantity THEN
|
|---|
| 99 | IF current_setting('eduberza.trade_fill', true) IS DISTINCT FROM 'on' THEN
|
|---|
| 100 | RAISE EXCEPTION 'filled quantity of order % can only change through a trade', OLD.id
|
|---|
| 101 | USING ERRCODE = 'check_violation';
|
|---|
| 102 | END IF;
|
|---|
| 103 | IF NEW.filled_quantity < OLD.filled_quantity THEN
|
|---|
| 104 | RAISE EXCEPTION 'filled quantity of order % cannot decrease', OLD.id
|
|---|
| 105 | USING ERRCODE = 'check_violation';
|
|---|
| 106 | END IF;
|
|---|
| 107 | END IF;
|
|---|
| 108 | IF NEW.status = 'cancelled' AND OLD.status <> 'cancelled' THEN
|
|---|
| 109 | IF NEW.filled_quantity <> OLD.filled_quantity THEN
|
|---|
| 110 | RAISE EXCEPTION 'an order cannot be filled and cancelled in the same step'
|
|---|
| 111 | USING ERRCODE = 'check_violation';
|
|---|
| 112 | END IF;
|
|---|
| 113 | NEW.executed_at := NULL;
|
|---|
| 114 | RETURN NEW;
|
|---|
| 115 | END IF;
|
|---|
| 116 | ELSE
|
|---|
| 117 | IF NOT EXISTS (SELECT 1 FROM project.markets WHERE id = NEW.market_id AND is_active) THEN
|
|---|
| 118 | RAISE EXCEPTION 'market % is not active; no new orders accepted', NEW.market_id
|
|---|
| 119 | USING ERRCODE = 'check_violation';
|
|---|
| 120 | END IF;
|
|---|
| 121 | IF NEW.price IS NULL OR NEW.price <= 0 THEN
|
|---|
| 122 | RAISE EXCEPTION 'an order needs a positive price (limit price, or the market price for a market order)'
|
|---|
| 123 | USING ERRCODE = 'check_violation';
|
|---|
| 124 | END IF;
|
|---|
| 125 | IF NEW.status = 'cancelled' THEN
|
|---|
| 126 | RAISE EXCEPTION 'an order cannot be created already cancelled'
|
|---|
| 127 | USING ERRCODE = 'check_violation';
|
|---|
| 128 | END IF;
|
|---|
| 129 | -- A new order starts unfilled. The only exception is importing an
|
|---|
| 130 | -- order that was completely executed in the past (sample data).
|
|---|
| 131 | IF NEW.filled_quantity NOT IN (0, NEW.quantity) THEN
|
|---|
| 132 | RAISE EXCEPTION 'a new order is either unfilled or (imported history) completely filled'
|
|---|
| 133 | USING ERRCODE = 'check_violation';
|
|---|
| 134 | END IF;
|
|---|
| 135 | END IF;
|
|---|
| 136 |
|
|---|
| 137 | v_derived := CASE
|
|---|
| 138 | WHEN NEW.filled_quantity = 0 THEN 'open'
|
|---|
| 139 | WHEN NEW.filled_quantity < NEW.quantity THEN 'partially_filled'
|
|---|
| 140 | ELSE 'executed'
|
|---|
| 141 | END;
|
|---|
| 142 | IF NEW.status IS DISTINCT FROM v_derived
|
|---|
| 143 | AND (TG_OP = 'INSERT' OR NEW.status IS DISTINCT FROM OLD.status) THEN
|
|---|
| 144 | RAISE EXCEPTION 'order status % does not match filled quantity % of %; status is derived automatically',
|
|---|
| 145 | NEW.status, NEW.filled_quantity, NEW.quantity
|
|---|
| 146 | USING ERRCODE = 'check_violation';
|
|---|
| 147 | END IF;
|
|---|
| 148 | NEW.status := v_derived;
|
|---|
| 149 |
|
|---|
| 150 | IF v_derived = 'executed' THEN
|
|---|
| 151 | NEW.executed_at := COALESCE(NEW.executed_at, now());
|
|---|
| 152 | ELSE
|
|---|
| 153 | NEW.executed_at := NULL;
|
|---|
| 154 | END IF;
|
|---|
| 155 | RETURN NEW;
|
|---|
| 156 | END $$;
|
|---|
| 157 |
|
|---|
| 158 | CREATE TRIGGER orders_lifecycle
|
|---|
| 159 | BEFORE INSERT OR UPDATE ON project.orders
|
|---|
| 160 | FOR EACH ROW EXECUTE FUNCTION project.trg_orders_lifecycle();
|
|---|
| 161 | ```
|
|---|
| 162 |
|
|---|
| 163 | Because the status is derived, the application never sets `open`, `partially_filled` or
|
|---|
| 164 | `executed` itself — it records trades, and the status follows.
|
|---|
| 165 |
|
|---|
| 166 | ## Trade consistency
|
|---|
| 167 |
|
|---|
| 168 | ### Data requirements description
|
|---|
| 169 |
|
|---|
| 170 | **Business rule.** A trade that fills orders must be possible for those orders:
|
|---|
| 171 |
|
|---|
| 172 | - the buy side is a buy order and the sell side is a sell order;
|
|---|
| 173 | - both are on the trade's market and still active — a cancelled or executed order can never
|
|---|
| 174 | trade again;
|
|---|
| 175 | - the trade quantity does not exceed the remaining quantity of either order;
|
|---|
| 176 | - the price respects both limits: at most the buy order's price, at least the sell order's price;
|
|---|
| 177 | - the two orders belong to different users (no self-trade).
|
|---|
| 178 |
|
|---|
| 179 | Recording the trade must fill both orders by exactly the traded quantity (and so move their
|
|---|
| 180 | status), and move the money and crypto for both users, all together. A trade that filled orders
|
|---|
| 181 | is history and can't be changed or deleted afterwards — the fills and the money moved would no
|
|---|
| 182 | longer match it.
|
|---|
| 183 |
|
|---|
| 184 | **Why it is non-trivial.** One trade row has to be checked against two other rows of another
|
|---|
| 185 | table (the orders), including their current remaining quantity, and a single insert must cause
|
|---|
| 186 | consistent changes in five tables (`market_trades`, both `orders`, both users' `users` and
|
|---|
| 187 | `holdings` rows, two `transactions` rows).
|
|---|
| 188 |
|
|---|
| 189 | **PostgreSQL features.**
|
|---|
| 190 |
|
|---|
| 191 | - `BEFORE INSERT` trigger `market_trades_validate` — the compatibility rules. It locks both
|
|---|
| 192 | orders (`FOR UPDATE`), so two concurrent trades can't both take the same remaining quantity.
|
|---|
| 193 | - `AFTER INSERT` trigger `market_trades_fill` — raises `filled_quantity` on both orders
|
|---|
| 194 | (automatic status change through requirement 1).
|
|---|
| 195 | - `BEFORE UPDATE OR DELETE` trigger `market_trades_immutable`.
|
|---|
| 196 | - Stored function `execute_trade` — the settlement of money and crypto for one trade.
|
|---|
| 197 |
|
|---|
| 198 | Because the rules sit on `market_trades` itself, even a trade inserted by hand is validated and
|
|---|
| 199 | fills the orders.
|
|---|
| 200 |
|
|---|
| 201 | **Tables affected.** `market_trades`, `orders`, `users`, `holdings`, `transactions`.
|
|---|
| 202 |
|
|---|
| 203 | ### Implementation
|
|---|
| 204 |
|
|---|
| 205 | #### Triggers
|
|---|
| 206 |
|
|---|
| 207 | ```sql
|
|---|
| 208 | CREATE OR REPLACE FUNCTION project.trg_market_trades_validate()
|
|---|
| 209 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 210 | DECLARE
|
|---|
| 211 | o project.orders%ROWTYPE;
|
|---|
| 212 | v_uid uuid;
|
|---|
| 213 | v_id uuid;
|
|---|
| 214 | v_role varchar(4);
|
|---|
| 215 | BEGIN
|
|---|
| 216 | IF NEW.buy_order_id IS NULL AND NEW.sell_order_id IS NULL THEN
|
|---|
| 217 | RETURN NEW;
|
|---|
| 218 | END IF;
|
|---|
| 219 | IF NEW.buy_order_id IS NOT DISTINCT FROM NEW.sell_order_id THEN
|
|---|
| 220 | RAISE EXCEPTION 'an order cannot trade with itself'
|
|---|
| 221 | USING ERRCODE = 'check_violation';
|
|---|
| 222 | END IF;
|
|---|
| 223 |
|
|---|
| 224 | FOREACH v_role IN ARRAY ARRAY['buy', 'sell'] LOOP
|
|---|
| 225 | v_id := CASE v_role WHEN 'buy' THEN NEW.buy_order_id ELSE NEW.sell_order_id END;
|
|---|
| 226 | CONTINUE WHEN v_id IS NULL;
|
|---|
| 227 |
|
|---|
| 228 | SELECT * INTO o FROM project.orders WHERE id = v_id FOR UPDATE;
|
|---|
| 229 | IF o.side <> v_role THEN
|
|---|
| 230 | RAISE EXCEPTION 'order % is a % order and cannot be the % side of a trade', v_id, o.side, v_role
|
|---|
| 231 | USING ERRCODE = 'check_violation';
|
|---|
| 232 | END IF;
|
|---|
| 233 | IF o.market_id <> NEW.market_id THEN
|
|---|
| 234 | RAISE EXCEPTION 'order % is on a different market than the trade', v_id
|
|---|
| 235 | USING ERRCODE = 'check_violation';
|
|---|
| 236 | END IF;
|
|---|
| 237 | IF o.status NOT IN ('open', 'partially_filled') THEN
|
|---|
| 238 | RAISE EXCEPTION 'order % is % and cannot trade', v_id, o.status
|
|---|
| 239 | USING ERRCODE = 'check_violation';
|
|---|
| 240 | END IF;
|
|---|
| 241 | IF NEW.quantity > o.quantity - o.filled_quantity THEN
|
|---|
| 242 | RAISE EXCEPTION 'trade quantity % exceeds the remaining quantity % of order %',
|
|---|
| 243 | NEW.quantity, o.quantity - o.filled_quantity, v_id
|
|---|
| 244 | USING ERRCODE = 'check_violation';
|
|---|
| 245 | END IF;
|
|---|
| 246 | IF v_uid IS NOT NULL AND v_uid = o.user_id THEN
|
|---|
| 247 | RAISE EXCEPTION 'a user cannot trade with their own order'
|
|---|
| 248 | USING ERRCODE = 'check_violation';
|
|---|
| 249 | END IF;
|
|---|
| 250 | IF v_role = 'buy' AND NEW.price > o.price OR v_role = 'sell' AND NEW.price < o.price THEN
|
|---|
| 251 | RAISE EXCEPTION 'trade price % is outside the limit % of % order %', NEW.price, o.price, v_role, v_id
|
|---|
| 252 | USING ERRCODE = 'check_violation';
|
|---|
| 253 | END IF;
|
|---|
| 254 | v_uid := o.user_id;
|
|---|
| 255 | END LOOP;
|
|---|
| 256 | RETURN NEW;
|
|---|
| 257 | END $$;
|
|---|
| 258 |
|
|---|
| 259 | CREATE TRIGGER market_trades_validate
|
|---|
| 260 | BEFORE INSERT ON project.market_trades
|
|---|
| 261 | FOR EACH ROW EXECUTE FUNCTION project.trg_market_trades_validate();
|
|---|
| 262 | ```
|
|---|
| 263 |
|
|---|
| 264 | ```sql
|
|---|
| 265 | CREATE OR REPLACE FUNCTION project.trg_market_trades_fill()
|
|---|
| 266 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 267 | BEGIN
|
|---|
| 268 | IF NEW.buy_order_id IS NULL AND NEW.sell_order_id IS NULL THEN
|
|---|
| 269 | RETURN NULL;
|
|---|
| 270 | END IF;
|
|---|
| 271 | PERFORM set_config('eduberza.trade_fill', 'on', true);
|
|---|
| 272 | PERFORM set_config('eduberza.trade_price', NEW.price::text, true);
|
|---|
| 273 | UPDATE project.orders
|
|---|
| 274 | SET filled_quantity = filled_quantity + NEW.quantity,
|
|---|
| 275 | executed_at = CASE WHEN filled_quantity + NEW.quantity = quantity
|
|---|
| 276 | THEN NEW.executed_at END
|
|---|
| 277 | WHERE id IN (NEW.buy_order_id, NEW.sell_order_id);
|
|---|
| 278 | PERFORM set_config('eduberza.trade_fill', 'off', true);
|
|---|
| 279 | RETURN NULL;
|
|---|
| 280 | END $$;
|
|---|
| 281 |
|
|---|
| 282 | CREATE TRIGGER market_trades_fill
|
|---|
| 283 | AFTER INSERT ON project.market_trades
|
|---|
| 284 | FOR EACH ROW EXECUTE FUNCTION project.trg_market_trades_fill();
|
|---|
| 285 | ```
|
|---|
| 286 |
|
|---|
| 287 | ```sql
|
|---|
| 288 | CREATE OR REPLACE FUNCTION project.trg_market_trades_immutable()
|
|---|
| 289 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 290 | BEGIN
|
|---|
| 291 | IF OLD.buy_order_id IS NULL AND OLD.sell_order_id IS NULL THEN
|
|---|
| 292 | -- a simulated tick filled no order; allow it unless it is being
|
|---|
| 293 | -- turned into one that did
|
|---|
| 294 | IF TG_OP = 'DELETE' THEN
|
|---|
| 295 | RETURN OLD;
|
|---|
| 296 | ELSIF NEW.buy_order_id IS NULL AND NEW.sell_order_id IS NULL THEN
|
|---|
| 297 | RETURN NEW;
|
|---|
| 298 | END IF;
|
|---|
| 299 | END IF;
|
|---|
| 300 | RAISE EXCEPTION 'a trade that filled orders cannot be changed or deleted'
|
|---|
| 301 | USING ERRCODE = 'check_violation';
|
|---|
| 302 | END $$;
|
|---|
| 303 |
|
|---|
| 304 | CREATE TRIGGER market_trades_immutable
|
|---|
| 305 | BEFORE UPDATE OR DELETE ON project.market_trades
|
|---|
| 306 | FOR EACH ROW EXECUTE FUNCTION project.trg_market_trades_immutable();
|
|---|
| 307 | ```
|
|---|
| 308 |
|
|---|
| 309 | #### Stored procedures/functions
|
|---|
| 310 |
|
|---|
| 311 | `execute_trade(buy_order, sell_order, quantity, price)` — one trade. Either order may be `NULL`
|
|---|
| 312 | when the simulated market is the counterparty. The `INSERT` into `market_trades` validates the
|
|---|
| 313 | trade and fills the orders (triggers above); the function then settles both users:
|
|---|
| 314 |
|
|---|
| 315 | - **Buyer:** the reservation for the filled part is released. The buyer pays the actual cost
|
|---|
| 316 | (trade price × quantity); if the trade price is below the order's limit, the difference goes
|
|---|
| 317 | back to `available_balance`. The crypto is added to the holding at a running weighted average
|
|---|
| 318 | price, and a `buy` ledger row is written.
|
|---|
| 319 | - **Seller:** the reserved crypto is delivered out of the holding, the proceeds are credited, the
|
|---|
| 320 | cost basis is removed from `invested_balance`, and a `sell` ledger row is written.
|
|---|
| 321 |
|
|---|
| 322 | Both orders are locked in a fixed order (by id), so two trades on the same pair of orders can't
|
|---|
| 323 | deadlock.
|
|---|
| 324 |
|
|---|
| 325 | ```sql
|
|---|
| 326 | CREATE OR REPLACE FUNCTION project.execute_trade(
|
|---|
| 327 | p_buy_order uuid, p_sell_order uuid, p_quantity numeric, p_price numeric,
|
|---|
| 328 | p_aggressor varchar DEFAULT NULL)
|
|---|
| 329 | RETURNS bigint LANGUAGE plpgsql AS $$
|
|---|
| 330 | DECLARE
|
|---|
| 331 | b project.orders%ROWTYPE;
|
|---|
| 332 | s project.orders%ROWTYPE;
|
|---|
| 333 | v_market project.markets%ROWTYPE;
|
|---|
| 334 | v_trade_id bigint;
|
|---|
| 335 | v_release numeric;
|
|---|
| 336 | v_cost numeric;
|
|---|
| 337 | v_proceeds numeric;
|
|---|
| 338 | v_avg numeric;
|
|---|
| 339 | BEGIN
|
|---|
| 340 | IF p_buy_order IS NULL AND p_sell_order IS NULL THEN
|
|---|
| 341 | RAISE EXCEPTION 'a trade needs at least one order' USING ERRCODE = 'check_violation';
|
|---|
| 342 | END IF;
|
|---|
| 343 |
|
|---|
| 344 | -- lock both orders in a fixed order (by id) so two concurrent trades on
|
|---|
| 345 | -- the same pair of orders cannot deadlock
|
|---|
| 346 | PERFORM 1 FROM project.orders WHERE id IN (p_buy_order, p_sell_order) ORDER BY id FOR UPDATE;
|
|---|
| 347 | SELECT * INTO b FROM project.orders WHERE id = p_buy_order;
|
|---|
| 348 | SELECT * INTO s FROM project.orders WHERE id = p_sell_order;
|
|---|
| 349 | SELECT * INTO v_market FROM project.markets WHERE id = COALESCE(b.market_id, s.market_id);
|
|---|
| 350 |
|
|---|
| 351 | INSERT INTO project.market_trades
|
|---|
| 352 | (market_id, executed_at, price, quantity, side, source, buy_order_id, sell_order_id)
|
|---|
| 353 | VALUES (v_market.id, now(), p_price, p_quantity,
|
|---|
| 354 | COALESCE(p_aggressor, CASE WHEN p_sell_order IS NULL THEN 'buy' ELSE 'sell' END),
|
|---|
| 355 | CASE WHEN p_buy_order IS NOT NULL AND p_sell_order IS NOT NULL THEN 'match' ELSE 'market' END,
|
|---|
| 356 | p_buy_order, p_sell_order)
|
|---|
| 357 | RETURNING id INTO v_trade_id;
|
|---|
| 358 |
|
|---|
| 359 | IF p_buy_order IS NOT NULL THEN
|
|---|
| 360 | v_release := project.order_reservation(b.quantity - b.filled_quantity, b.price)
|
|---|
| 361 | - project.order_reservation(b.quantity - b.filled_quantity - p_quantity, b.price);
|
|---|
| 362 | v_cost := LEAST(round(p_quantity * p_price, 4), v_release);
|
|---|
| 363 |
|
|---|
| 364 | UPDATE project.users
|
|---|
| 365 | SET reserved_balance = reserved_balance - v_release,
|
|---|
| 366 | available_balance = available_balance + (v_release - v_cost),
|
|---|
| 367 | invested_balance = invested_balance + v_cost,
|
|---|
| 368 | updated_at = now()
|
|---|
| 369 | WHERE id = b.user_id;
|
|---|
| 370 |
|
|---|
| 371 | INSERT INTO project.holdings AS h (user_id, crypto_id, quantity, avg_price, updated_at)
|
|---|
| 372 | VALUES (b.user_id, v_market.crypto_id, p_quantity, p_price, now())
|
|---|
| 373 | ON CONFLICT (user_id, crypto_id) DO UPDATE
|
|---|
| 374 | SET avg_price = (h.quantity * h.avg_price + EXCLUDED.quantity * EXCLUDED.avg_price)
|
|---|
| 375 | / (h.quantity + EXCLUDED.quantity),
|
|---|
| 376 | quantity = h.quantity + EXCLUDED.quantity,
|
|---|
| 377 | updated_at = now();
|
|---|
| 378 |
|
|---|
| 379 | INSERT INTO project.transactions (user_id, type, amount, currency, related_order, description)
|
|---|
| 380 | VALUES (b.user_id, 'buy', -v_cost, v_market.quote_currency, b.id,
|
|---|
| 381 | format('Buy %s @ %s (trade %s)', p_quantity, p_price, v_trade_id));
|
|---|
| 382 | END IF;
|
|---|
| 383 |
|
|---|
| 384 | IF p_sell_order IS NOT NULL THEN
|
|---|
| 385 | SELECT avg_price INTO v_avg FROM project.holdings
|
|---|
| 386 | WHERE user_id = s.user_id AND crypto_id = v_market.crypto_id FOR UPDATE;
|
|---|
| 387 |
|
|---|
| 388 | UPDATE project.holdings
|
|---|
| 389 | SET quantity = quantity - p_quantity,
|
|---|
| 390 | reserved_quantity = reserved_quantity - p_quantity,
|
|---|
| 391 | updated_at = now()
|
|---|
| 392 | WHERE user_id = s.user_id AND crypto_id = v_market.crypto_id;
|
|---|
| 393 |
|
|---|
| 394 | v_proceeds := round(p_quantity * p_price, 4);
|
|---|
| 395 | UPDATE project.users
|
|---|
| 396 | SET available_balance = available_balance + v_proceeds,
|
|---|
| 397 | invested_balance = GREATEST(invested_balance - round(p_quantity * v_avg, 4), 0),
|
|---|
| 398 | updated_at = now()
|
|---|
| 399 | WHERE id = s.user_id;
|
|---|
| 400 |
|
|---|
| 401 | INSERT INTO project.transactions (user_id, type, amount, currency, related_order, description)
|
|---|
| 402 | VALUES (s.user_id, 'sell', v_proceeds, v_market.quote_currency, s.id,
|
|---|
| 403 | format('Sell %s @ %s (trade %s)', p_quantity, p_price, v_trade_id));
|
|---|
| 404 | END IF;
|
|---|
| 405 |
|
|---|
| 406 | RETURN v_trade_id;
|
|---|
| 407 | END $$;
|
|---|
| 408 | ```
|
|---|
| 409 |
|
|---|
| 410 | ## Balance and reservation consistency
|
|---|
| 411 |
|
|---|
| 412 | ### Data requirements description
|
|---|
| 413 |
|
|---|
| 414 | **Business rule.**
|
|---|
| 415 |
|
|---|
| 416 | 1. **Reserved cash matches active buy orders.** A user's `reserved_balance` always equals what
|
|---|
| 417 | their active buy orders still reserve: remaining quantity × order price, summed.
|
|---|
| 418 | 2. **Reserved crypto matches active sell orders.** A user's `holdings.reserved_quantity` for a
|
|---|
| 419 | crypto always equals the remaining quantity of their active sell orders for it.
|
|---|
| 420 | 3. **Cash matches the ledger.** `available_balance + reserved_balance` always equals the sum of
|
|---|
| 421 | the user's ledger (`transactions`). Reserving only moves cash between the two columns; money
|
|---|
| 422 | actually arrives or leaves only with a ledger row (deposit, buy fill, sell fill).
|
|---|
| 423 |
|
|---|
| 424 | Together these make it impossible for an order operation to leave a balance in an inconsistent
|
|---|
| 425 | state: money or crypto reserved for nothing, an order that is not backed by a reservation (and
|
|---|
| 426 | so could spend the same money twice), or cash that appeared or vanished without a ledger entry.
|
|---|
| 427 |
|
|---|
| 428 | **Why it is non-trivial.** Each rule is an equality between a column and an aggregate over
|
|---|
| 429 | *other* rows of *other* tables. Worse, every legitimate operation breaks it for a moment:
|
|---|
| 430 | placing a buy moves cash to `reserved_balance` in one statement and inserts the order in the
|
|---|
| 431 | next; a trade releases the reservation, pays, and writes the ledger in several statements. Only
|
|---|
| 432 | the state at the end of the transaction has to be consistent.
|
|---|
| 433 |
|
|---|
| 434 | **PostgreSQL feature.** `CONSTRAINT TRIGGER … DEFERRABLE INITIALLY DEFERRED` — row triggers
|
|---|
| 435 | whose check runs at `COMMIT`, on the final state. A transaction that leaves any of the three
|
|---|
| 436 | equalities broken fails at `COMMIT` and is rolled back as a whole. Each rule has a trigger on
|
|---|
| 437 | every table whose change can break it. `order_reservation(remaining, price)` is the one place
|
|---|
| 438 | that defines how a reservation is rounded, so placing, filling, cancelling and checking always
|
|---|
| 439 | agree to the last decimal.
|
|---|
| 440 |
|
|---|
| 441 | **Tables affected.** `users`, `holdings`, `orders`, `transactions`.
|
|---|
| 442 |
|
|---|
| 443 | ### Implementation
|
|---|
| 444 |
|
|---|
| 445 | #### Stored procedures/functions
|
|---|
| 446 |
|
|---|
| 447 | ```sql
|
|---|
| 448 | CREATE OR REPLACE FUNCTION project.order_reservation(p_remaining numeric, p_price numeric)
|
|---|
| 449 | RETURNS numeric LANGUAGE sql IMMUTABLE AS $$
|
|---|
| 450 | SELECT round(p_remaining * p_price, 4)
|
|---|
| 451 | $$;
|
|---|
| 452 | ```
|
|---|
| 453 |
|
|---|
| 454 | #### Triggers
|
|---|
| 455 |
|
|---|
| 456 | ```sql
|
|---|
| 457 | CREATE OR REPLACE FUNCTION project.trg_reserved_cash_matches_orders()
|
|---|
| 458 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 459 | DECLARE
|
|---|
| 460 | v_user uuid := CASE WHEN TG_TABLE_NAME = 'users' THEN NEW.id END;
|
|---|
| 461 | v_reserved numeric;
|
|---|
| 462 | v_needed numeric;
|
|---|
| 463 | BEGIN
|
|---|
| 464 | IF TG_TABLE_NAME = 'orders' THEN
|
|---|
| 465 | v_user := NEW.user_id;
|
|---|
| 466 | END IF;
|
|---|
| 467 | SELECT reserved_balance INTO v_reserved FROM project.users WHERE id = v_user;
|
|---|
| 468 | IF NOT FOUND THEN
|
|---|
| 469 | RETURN NULL;
|
|---|
| 470 | END IF;
|
|---|
| 471 | SELECT COALESCE(SUM(project.order_reservation(quantity - filled_quantity, price)), 0)
|
|---|
| 472 | INTO v_needed
|
|---|
| 473 | FROM project.orders
|
|---|
| 474 | WHERE user_id = v_user AND side = 'buy' AND status IN ('open', 'partially_filled');
|
|---|
| 475 | IF v_reserved <> v_needed THEN
|
|---|
| 476 | RAISE EXCEPTION 'reserved balance % does not match the % needed by active buy orders (user %)',
|
|---|
| 477 | v_reserved, v_needed, v_user
|
|---|
| 478 | USING ERRCODE = 'check_violation', CONSTRAINT = 'reserved_cash_matches_orders';
|
|---|
| 479 | END IF;
|
|---|
| 480 | RETURN NULL;
|
|---|
| 481 | END $$;
|
|---|
| 482 | ```
|
|---|
| 483 |
|
|---|
| 484 | ```sql
|
|---|
| 485 | CREATE OR REPLACE FUNCTION project.trg_reserved_crypto_matches_orders()
|
|---|
| 486 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 487 | DECLARE
|
|---|
| 488 | v_user uuid;
|
|---|
| 489 | v_crypto uuid;
|
|---|
| 490 | v_reserved numeric;
|
|---|
| 491 | v_needed numeric;
|
|---|
| 492 | BEGIN
|
|---|
| 493 | IF TG_TABLE_NAME = 'holdings' THEN
|
|---|
| 494 | v_user := NEW.user_id;
|
|---|
| 495 | v_crypto := NEW.crypto_id;
|
|---|
| 496 | ELSE
|
|---|
| 497 | IF NEW.side <> 'sell' THEN
|
|---|
| 498 | RETURN NULL;
|
|---|
| 499 | END IF;
|
|---|
| 500 | v_user := NEW.user_id;
|
|---|
| 501 | SELECT crypto_id INTO v_crypto FROM project.markets WHERE id = NEW.market_id;
|
|---|
| 502 | END IF;
|
|---|
| 503 | SELECT COALESCE(SUM(reserved_quantity), 0) INTO v_reserved
|
|---|
| 504 | FROM project.holdings WHERE user_id = v_user AND crypto_id = v_crypto;
|
|---|
| 505 | SELECT COALESCE(SUM(o.quantity - o.filled_quantity), 0) INTO v_needed
|
|---|
| 506 | FROM project.orders o
|
|---|
| 507 | JOIN project.markets m ON m.id = o.market_id
|
|---|
| 508 | WHERE o.user_id = v_user AND m.crypto_id = v_crypto
|
|---|
| 509 | AND o.side = 'sell' AND o.status IN ('open', 'partially_filled');
|
|---|
| 510 | IF v_reserved <> v_needed THEN
|
|---|
| 511 | RAISE EXCEPTION 'reserved quantity % does not match the % needed by active sell orders (user %, crypto %)',
|
|---|
| 512 | v_reserved, v_needed, v_user, v_crypto
|
|---|
| 513 | USING ERRCODE = 'check_violation', CONSTRAINT = 'reserved_crypto_matches_orders';
|
|---|
| 514 | END IF;
|
|---|
| 515 | RETURN NULL;
|
|---|
| 516 | END $$;
|
|---|
| 517 | ```
|
|---|
| 518 |
|
|---|
| 519 | ```sql
|
|---|
| 520 | CREATE OR REPLACE FUNCTION project.trg_cash_matches_ledger()
|
|---|
| 521 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 522 | DECLARE
|
|---|
| 523 | v_user uuid;
|
|---|
| 524 | v_cash numeric;
|
|---|
| 525 | v_ledger numeric;
|
|---|
| 526 | BEGIN
|
|---|
| 527 | IF TG_TABLE_NAME = 'users' THEN
|
|---|
| 528 | v_user := NEW.id;
|
|---|
| 529 | ELSIF TG_OP = 'DELETE' THEN
|
|---|
| 530 | v_user := OLD.user_id;
|
|---|
| 531 | ELSE
|
|---|
| 532 | v_user := NEW.user_id;
|
|---|
| 533 | END IF;
|
|---|
| 534 | SELECT available_balance + reserved_balance INTO v_cash FROM project.users WHERE id = v_user;
|
|---|
| 535 | IF NOT FOUND THEN
|
|---|
| 536 | RETURN NULL;
|
|---|
| 537 | END IF;
|
|---|
| 538 | SELECT COALESCE(SUM(amount), 0) INTO v_ledger FROM project.transactions WHERE user_id = v_user;
|
|---|
| 539 | IF v_cash <> v_ledger THEN
|
|---|
| 540 | RAISE EXCEPTION 'cash % (available + reserved) does not match the ledger total % (user %)',
|
|---|
| 541 | v_cash, v_ledger, v_user
|
|---|
| 542 | USING ERRCODE = 'check_violation', CONSTRAINT = 'cash_matches_ledger';
|
|---|
| 543 | END IF;
|
|---|
| 544 | RETURN NULL;
|
|---|
| 545 | END $$;
|
|---|
| 546 | ```
|
|---|
| 547 |
|
|---|
| 548 | ```sql
|
|---|
| 549 | CREATE INDEX idx_orders_active ON project.orders (user_id, side)
|
|---|
| 550 | WHERE status IN ('open', 'partially_filled');
|
|---|
| 551 |
|
|---|
| 552 | CREATE CONSTRAINT TRIGGER reserved_cash_matches_orders
|
|---|
| 553 | AFTER INSERT OR UPDATE OF reserved_balance ON project.users
|
|---|
| 554 | DEFERRABLE INITIALLY DEFERRED
|
|---|
| 555 | FOR EACH ROW EXECUTE FUNCTION project.trg_reserved_cash_matches_orders();
|
|---|
| 556 |
|
|---|
| 557 | CREATE CONSTRAINT TRIGGER reserved_cash_matches_orders
|
|---|
| 558 | AFTER INSERT OR UPDATE OF status, filled_quantity ON project.orders
|
|---|
| 559 | DEFERRABLE INITIALLY DEFERRED
|
|---|
| 560 | FOR EACH ROW EXECUTE FUNCTION project.trg_reserved_cash_matches_orders();
|
|---|
| 561 |
|
|---|
| 562 | CREATE CONSTRAINT TRIGGER reserved_crypto_matches_orders
|
|---|
| 563 | AFTER INSERT OR UPDATE OF reserved_quantity ON project.holdings
|
|---|
| 564 | DEFERRABLE INITIALLY DEFERRED
|
|---|
| 565 | FOR EACH ROW EXECUTE FUNCTION project.trg_reserved_crypto_matches_orders();
|
|---|
| 566 |
|
|---|
| 567 | CREATE CONSTRAINT TRIGGER reserved_crypto_matches_orders
|
|---|
| 568 | AFTER INSERT OR UPDATE OF status, filled_quantity ON project.orders
|
|---|
| 569 | DEFERRABLE INITIALLY DEFERRED
|
|---|
| 570 | FOR EACH ROW EXECUTE FUNCTION project.trg_reserved_crypto_matches_orders();
|
|---|
| 571 |
|
|---|
| 572 | CREATE CONSTRAINT TRIGGER cash_matches_ledger
|
|---|
| 573 | AFTER INSERT OR UPDATE OF available_balance, reserved_balance ON project.users
|
|---|
| 574 | DEFERRABLE INITIALLY DEFERRED
|
|---|
| 575 | FOR EACH ROW EXECUTE FUNCTION project.trg_cash_matches_ledger();
|
|---|
| 576 |
|
|---|
| 577 | CREATE CONSTRAINT TRIGGER cash_matches_ledger
|
|---|
| 578 | AFTER INSERT OR UPDATE OR DELETE ON project.transactions
|
|---|
| 579 | DEFERRABLE INITIALLY DEFERRED
|
|---|
| 580 | FOR EACH ROW EXECUTE FUNCTION project.trg_cash_matches_ledger();
|
|---|
| 581 | ```
|
|---|
| 582 |
|
|---|
| 583 | The partial index `idx_orders_active` covers exactly the rows the reservation checks sum, and
|
|---|
| 584 | stays small because active orders are few compared with the whole order history.
|
|---|
| 585 |
|
|---|
| 586 | ## Placing and cancelling orders
|
|---|
| 587 |
|
|---|
| 588 | ### Data requirements description
|
|---|
| 589 |
|
|---|
| 590 | **Business rule.** Placing an order must, as one unit:
|
|---|
| 591 |
|
|---|
| 592 | 1. check the market, side, type, quantity and price;
|
|---|
| 593 | 2. reserve what the order commits — cash (quantity × price) for a buy, crypto for a sell — and
|
|---|
| 594 | refuse the order if not enough is free;
|
|---|
| 595 | 3. record the order;
|
|---|
| 596 | 4. match it against the order book: other users' active limit orders on the same market whose
|
|---|
| 597 | price is acceptable, best price first and oldest first (price–time priority), each trade at
|
|---|
| 598 | the resting order's price;
|
|---|
| 599 | 5. fill whatever is still unfilled from the simulated market if it is marketable at the current
|
|---|
| 600 | market price.
|
|---|
| 601 |
|
|---|
| 602 | A **market order** is priced at the current market price, so it always fills completely in step
|
|---|
| 603 | 4 or 5 and never waits. A **limit order** that isn't marketable stays in the order book.
|
|---|
| 604 | Cancelling an order must release exactly what it still reserves, and only for an active order of
|
|---|
| 605 | the caller.
|
|---|
| 606 |
|
|---|
| 607 | **Why it is non-trivial.** It is a multi-step operation over five tables whose steps depend on
|
|---|
| 608 | each other (how much is left after each match, what to release), and it has to be correct under
|
|---|
| 609 | concurrency: two orders of the same user must not both see the same free cash.
|
|---|
| 610 |
|
|---|
| 611 | **PostgreSQL feature.** Stored functions (PL/pgSQL). `place_order` locks the user's row
|
|---|
| 612 | (`SELECT … FOR UPDATE`) before checking free cash or crypto, so concurrent orders of the same
|
|---|
| 613 | user are serialised. Every step's consistency is still checked by the triggers of requirements
|
|---|
| 614 | 1–3.
|
|---|
| 615 |
|
|---|
| 616 | **Tables affected.** `orders`, `users`, `holdings`, `market_trades`, `transactions`.
|
|---|
| 617 |
|
|---|
| 618 | ### Implementation
|
|---|
| 619 |
|
|---|
| 620 | #### Stored procedures/functions
|
|---|
| 621 |
|
|---|
| 622 | ```sql
|
|---|
| 623 | CREATE OR REPLACE FUNCTION project.latest_price(p_market_id uuid)
|
|---|
| 624 | RETURNS numeric LANGUAGE sql STABLE AS $$
|
|---|
| 625 | SELECT price FROM project.market_trades
|
|---|
| 626 | WHERE market_id = p_market_id
|
|---|
| 627 | ORDER BY executed_at DESC, id DESC
|
|---|
| 628 | LIMIT 1
|
|---|
| 629 | $$;
|
|---|
| 630 | ```
|
|---|
| 631 |
|
|---|
| 632 | ```sql
|
|---|
| 633 | CREATE OR REPLACE FUNCTION project.match_order(p_order_id uuid)
|
|---|
| 634 | RETURNS int LANGUAGE plpgsql AS $$
|
|---|
| 635 | DECLARE
|
|---|
| 636 | o project.orders%ROWTYPE;
|
|---|
| 637 | r record;
|
|---|
| 638 | v_rem numeric;
|
|---|
| 639 | v_qty numeric;
|
|---|
| 640 | v_count int := 0;
|
|---|
| 641 | BEGIN
|
|---|
| 642 | SELECT * INTO o FROM project.orders WHERE id = p_order_id FOR UPDATE;
|
|---|
| 643 | FOR r IN
|
|---|
| 644 | SELECT id, price, quantity - filled_quantity AS remaining
|
|---|
| 645 | FROM project.orders
|
|---|
| 646 | WHERE market_id = o.market_id
|
|---|
| 647 | AND side <> o.side
|
|---|
| 648 | AND type = 'limit'
|
|---|
| 649 | AND status IN ('open', 'partially_filled')
|
|---|
| 650 | AND user_id <> o.user_id
|
|---|
| 651 | AND (o.side = 'buy' AND price <= o.price
|
|---|
| 652 | OR o.side = 'sell' AND price >= o.price)
|
|---|
| 653 | ORDER BY CASE WHEN o.side = 'buy' THEN price END ASC,
|
|---|
| 654 | CASE WHEN o.side = 'sell' THEN price END DESC,
|
|---|
| 655 | placed_at, id
|
|---|
| 656 | FOR UPDATE
|
|---|
| 657 | LOOP
|
|---|
| 658 | SELECT quantity - filled_quantity INTO v_rem FROM project.orders WHERE id = p_order_id;
|
|---|
| 659 | EXIT WHEN v_rem = 0;
|
|---|
| 660 | v_qty := LEAST(v_rem, r.remaining);
|
|---|
| 661 | IF o.side = 'buy' THEN
|
|---|
| 662 | PERFORM project.execute_trade(o.id, r.id, v_qty, r.price, 'buy');
|
|---|
| 663 | ELSE
|
|---|
| 664 | PERFORM project.execute_trade(r.id, o.id, v_qty, r.price, 'sell');
|
|---|
| 665 | END IF;
|
|---|
| 666 | v_count := v_count + 1;
|
|---|
| 667 | END LOOP;
|
|---|
| 668 | RETURN v_count;
|
|---|
| 669 | END $$;
|
|---|
| 670 | ```
|
|---|
| 671 |
|
|---|
| 672 | ```sql
|
|---|
| 673 | CREATE OR REPLACE FUNCTION project.place_order(
|
|---|
| 674 | p_user_id uuid, p_market_id uuid, p_side varchar, p_type varchar,
|
|---|
| 675 | p_quantity numeric, p_limit_price numeric DEFAULT NULL)
|
|---|
| 676 | RETURNS uuid LANGUAGE plpgsql AS $$
|
|---|
| 677 | DECLARE
|
|---|
| 678 | v_market numeric := project.latest_price(p_market_id);
|
|---|
| 679 | v_price numeric;
|
|---|
| 680 | v_available numeric;
|
|---|
| 681 | v_free numeric;
|
|---|
| 682 | v_crypto uuid;
|
|---|
| 683 | v_order uuid;
|
|---|
| 684 | v_rem numeric;
|
|---|
| 685 | BEGIN
|
|---|
| 686 | IF p_side NOT IN ('buy', 'sell') OR p_type NOT IN ('market', 'limit') THEN
|
|---|
| 687 | RAISE EXCEPTION 'invalid side % or type %', p_side, p_type USING ERRCODE = 'check_violation';
|
|---|
| 688 | END IF;
|
|---|
| 689 | IF p_quantity IS NULL OR p_quantity <= 0 THEN
|
|---|
| 690 | RAISE EXCEPTION 'quantity must be positive' USING ERRCODE = 'check_violation';
|
|---|
| 691 | END IF;
|
|---|
| 692 | IF p_type = 'limit' THEN
|
|---|
| 693 | IF p_limit_price IS NULL OR p_limit_price <= 0 THEN
|
|---|
| 694 | RAISE EXCEPTION 'a limit order needs a positive limit price' USING ERRCODE = 'check_violation';
|
|---|
| 695 | END IF;
|
|---|
| 696 | v_price := p_limit_price;
|
|---|
| 697 | ELSE
|
|---|
| 698 | IF v_market IS NULL THEN
|
|---|
| 699 | RAISE EXCEPTION 'market has no price yet' USING ERRCODE = 'check_violation';
|
|---|
| 700 | END IF;
|
|---|
| 701 | v_price := v_market;
|
|---|
| 702 | END IF;
|
|---|
| 703 |
|
|---|
| 704 | SELECT available_balance INTO v_available FROM project.users WHERE id = p_user_id FOR UPDATE;
|
|---|
| 705 | IF NOT FOUND THEN
|
|---|
| 706 | RAISE EXCEPTION 'user % does not exist', p_user_id USING ERRCODE = 'no_data_found';
|
|---|
| 707 | END IF;
|
|---|
| 708 |
|
|---|
| 709 | IF p_side = 'buy' THEN
|
|---|
| 710 | IF v_available < project.order_reservation(p_quantity, v_price) THEN
|
|---|
| 711 | RAISE EXCEPTION 'insufficient funds: the order needs %, available %',
|
|---|
| 712 | project.order_reservation(p_quantity, v_price), v_available
|
|---|
| 713 | USING ERRCODE = 'check_violation';
|
|---|
| 714 | END IF;
|
|---|
| 715 | UPDATE project.users
|
|---|
| 716 | SET available_balance = available_balance - project.order_reservation(p_quantity, v_price),
|
|---|
| 717 | reserved_balance = reserved_balance + project.order_reservation(p_quantity, v_price),
|
|---|
| 718 | updated_at = now()
|
|---|
| 719 | WHERE id = p_user_id;
|
|---|
| 720 | ELSE
|
|---|
| 721 | SELECT crypto_id INTO v_crypto FROM project.markets WHERE id = p_market_id;
|
|---|
| 722 | SELECT quantity - reserved_quantity INTO v_free FROM project.holdings
|
|---|
| 723 | WHERE user_id = p_user_id AND crypto_id = v_crypto FOR UPDATE;
|
|---|
| 724 | IF COALESCE(v_free, 0) < p_quantity THEN
|
|---|
| 725 | RAISE EXCEPTION 'insufficient holding: trying to sell %, free to sell %', p_quantity, COALESCE(v_free, 0)
|
|---|
| 726 | USING ERRCODE = 'check_violation';
|
|---|
| 727 | END IF;
|
|---|
| 728 | UPDATE project.holdings
|
|---|
| 729 | SET reserved_quantity = reserved_quantity + p_quantity, updated_at = now()
|
|---|
| 730 | WHERE user_id = p_user_id AND crypto_id = v_crypto;
|
|---|
| 731 | END IF;
|
|---|
| 732 |
|
|---|
| 733 | INSERT INTO project.orders (user_id, market_id, side, type, status, quantity, price)
|
|---|
| 734 | VALUES (p_user_id, p_market_id, p_side, p_type, 'open', p_quantity, v_price)
|
|---|
| 735 | RETURNING id INTO v_order;
|
|---|
| 736 |
|
|---|
| 737 | PERFORM project.match_order(v_order);
|
|---|
| 738 |
|
|---|
| 739 | SELECT quantity - filled_quantity INTO v_rem FROM project.orders WHERE id = v_order;
|
|---|
| 740 | IF v_rem > 0 AND v_market IS NOT NULL
|
|---|
| 741 | AND (p_side = 'buy' AND v_market <= v_price OR p_side = 'sell' AND v_market >= v_price) THEN
|
|---|
| 742 | IF p_side = 'buy' THEN
|
|---|
| 743 | PERFORM project.execute_trade(v_order, NULL, v_rem, v_market, 'buy');
|
|---|
| 744 | ELSE
|
|---|
| 745 | PERFORM project.execute_trade(NULL, v_order, v_rem, v_market, 'sell');
|
|---|
| 746 | END IF;
|
|---|
| 747 | END IF;
|
|---|
| 748 | RETURN v_order;
|
|---|
| 749 | END $$;
|
|---|
| 750 | ```
|
|---|
| 751 |
|
|---|
| 752 | ```sql
|
|---|
| 753 | CREATE OR REPLACE FUNCTION project.cancel_order(p_order_id uuid, p_user_id uuid DEFAULT NULL)
|
|---|
| 754 | RETURNS void LANGUAGE plpgsql AS $$
|
|---|
| 755 | DECLARE
|
|---|
| 756 | o project.orders%ROWTYPE;
|
|---|
| 757 | v_rem numeric;
|
|---|
| 758 | BEGIN
|
|---|
| 759 | SELECT * INTO o FROM project.orders WHERE id = p_order_id FOR UPDATE;
|
|---|
| 760 | IF NOT FOUND THEN
|
|---|
| 761 | RAISE EXCEPTION 'order % does not exist', p_order_id USING ERRCODE = 'no_data_found';
|
|---|
| 762 | END IF;
|
|---|
| 763 | IF p_user_id IS NOT NULL AND o.user_id <> p_user_id THEN
|
|---|
| 764 | RAISE EXCEPTION 'order % does not belong to this user', p_order_id
|
|---|
| 765 | USING ERRCODE = 'insufficient_privilege';
|
|---|
| 766 | END IF;
|
|---|
| 767 | IF o.status NOT IN ('open', 'partially_filled') THEN
|
|---|
| 768 | RAISE EXCEPTION 'order % is % and cannot be cancelled', p_order_id, o.status
|
|---|
| 769 | USING ERRCODE = 'check_violation';
|
|---|
| 770 | END IF;
|
|---|
| 771 |
|
|---|
| 772 | v_rem := o.quantity - o.filled_quantity;
|
|---|
| 773 | IF o.side = 'buy' THEN
|
|---|
| 774 | UPDATE project.users
|
|---|
| 775 | SET reserved_balance = reserved_balance - project.order_reservation(v_rem, o.price),
|
|---|
| 776 | available_balance = available_balance + project.order_reservation(v_rem, o.price),
|
|---|
| 777 | updated_at = now()
|
|---|
| 778 | WHERE id = o.user_id;
|
|---|
| 779 | ELSE
|
|---|
| 780 | UPDATE project.holdings h
|
|---|
| 781 | SET reserved_quantity = h.reserved_quantity - v_rem, updated_at = now()
|
|---|
| 782 | FROM project.markets m
|
|---|
| 783 | WHERE m.id = o.market_id AND h.user_id = o.user_id AND h.crypto_id = m.crypto_id;
|
|---|
| 784 | END IF;
|
|---|
| 785 |
|
|---|
| 786 | UPDATE project.orders SET status = 'cancelled' WHERE id = p_order_id;
|
|---|
| 787 | END $$;
|
|---|
| 788 | ```
|
|---|
| 789 |
|
|---|
| 790 | In the prototype, placing an order is now a single call:
|
|---|
| 791 |
|
|---|
| 792 | ```go
|
|---|
| 793 | err = db.DB.QueryRow(
|
|---|
| 794 | `SELECT place_order($1, $2, $3, $4, $5, $6)`,
|
|---|
| 795 | s.UserID, m.ID, side, orderType, qty, limit.value(),
|
|---|
| 796 | ).Scan(&orderID)
|
|---|
| 797 | ```
|
|---|
| 798 |
|
|---|
| 799 | Cancelling is `SELECT cancel_order($1, $2)` with the order the user picked from a numbered list
|
|---|
| 800 | of their open orders ([`server/trade.go`](../../server/trade.go)).
|
|---|
| 801 |
|
|---|
| 802 | ## Automatic recording of order events
|
|---|
| 803 |
|
|---|
| 804 | ### Data requirements description
|
|---|
| 805 |
|
|---|
| 806 | **Business rule.** Every important thing that happens to an order is recorded with its time:
|
|---|
| 807 | placement, each fill (with the quantity filled and the trade price), and cancellation. This is
|
|---|
| 808 | the order's audit trail — the history of *how* it reached its current state, which the order row
|
|---|
| 809 | alone (only the current state) cannot show.
|
|---|
| 810 |
|
|---|
| 811 | **Why it is non-trivial.** Events come from several places — `place_order`, trades made by
|
|---|
| 812 | `match_order`, trades made by the background job, `cancel_order`, and any direct SQL. Recording
|
|---|
| 813 | them in each of those places would miss some; recording them where the change actually happens
|
|---|
| 814 | cannot.
|
|---|
| 815 |
|
|---|
| 816 | **PostgreSQL feature.** `AFTER INSERT OR UPDATE` row trigger on `orders`, writing into the new
|
|---|
| 817 | table `order_events`. The fill price is handed over by the trade trigger through a
|
|---|
| 818 | transaction-local setting.
|
|---|
| 819 |
|
|---|
| 820 | **Tables affected.** `order_events` (new), written from changes to `orders`.
|
|---|
| 821 |
|
|---|
| 822 | ### Implementation
|
|---|
| 823 |
|
|---|
| 824 | ```sql
|
|---|
| 825 | CREATE TABLE project.order_events (
|
|---|
| 826 | id bigserial PRIMARY KEY,
|
|---|
| 827 | order_id uuid NOT NULL REFERENCES project.orders(id) ON DELETE CASCADE,
|
|---|
| 828 | event_type varchar(20) NOT NULL
|
|---|
| 829 | CHECK (event_type IN ('placed', 'partially_filled', 'filled', 'cancelled')),
|
|---|
| 830 | quantity numeric(20,4) NOT NULL,
|
|---|
| 831 | price numeric(18,6),
|
|---|
| 832 | status_after varchar(20) NOT NULL,
|
|---|
| 833 | created_at timestamptz NOT NULL DEFAULT clock_timestamp()
|
|---|
| 834 | );
|
|---|
| 835 | ```
|
|---|
| 836 |
|
|---|
| 837 | #### Triggers
|
|---|
| 838 |
|
|---|
| 839 | ```sql
|
|---|
| 840 | CREATE OR REPLACE FUNCTION project.trg_orders_events()
|
|---|
| 841 | RETURNS trigger LANGUAGE plpgsql AS $$
|
|---|
| 842 | BEGIN
|
|---|
| 843 | IF TG_OP = 'INSERT' THEN
|
|---|
| 844 | INSERT INTO project.order_events (order_id, event_type, quantity, price, status_after)
|
|---|
| 845 | VALUES (NEW.id, 'placed', NEW.quantity, NEW.price, NEW.status);
|
|---|
| 846 | ELSIF NEW.filled_quantity > OLD.filled_quantity THEN
|
|---|
| 847 | INSERT INTO project.order_events (order_id, event_type, quantity, price, status_after)
|
|---|
| 848 | VALUES (NEW.id,
|
|---|
| 849 | CASE WHEN NEW.status = 'executed' THEN 'filled' ELSE 'partially_filled' END,
|
|---|
| 850 | NEW.filled_quantity - OLD.filled_quantity,
|
|---|
| 851 | current_setting('eduberza.trade_price', true)::numeric,
|
|---|
| 852 | NEW.status);
|
|---|
| 853 | ELSIF NEW.status = 'cancelled' AND OLD.status <> 'cancelled' THEN
|
|---|
| 854 | INSERT INTO project.order_events (order_id, event_type, quantity, price, status_after)
|
|---|
| 855 | VALUES (NEW.id, 'cancelled', NEW.quantity - NEW.filled_quantity, NEW.price, NEW.status);
|
|---|
| 856 | END IF;
|
|---|
| 857 | RETURN NULL;
|
|---|
| 858 | END $$;
|
|---|
| 859 |
|
|---|
| 860 | CREATE TRIGGER orders_events
|
|---|
| 861 | AFTER INSERT OR UPDATE ON project.orders
|
|---|
| 862 | FOR EACH ROW EXECUTE FUNCTION project.trg_orders_events();
|
|---|
| 863 | ```
|
|---|
| 864 |
|
|---|
| 865 | ## Views for derived trading data
|
|---|
| 866 |
|
|---|
| 867 | ### Data requirements description
|
|---|
| 868 |
|
|---|
| 869 | The application needs several things that are *derived* from orders, trades and balances. They
|
|---|
| 870 | are defined once as views, instead of repeating the calculations in the application code:
|
|---|
| 871 |
|
|---|
| 872 | | View | Derived data | Used by |
|
|---|
| 873 | |---|---|---|
|
|---|
| 874 | | `v_active_orders` | active orders with remaining quantity and what each one holds in reserve | CLI `[13] My open orders`, `[14] Cancel an order` |
|
|---|
| 875 | | `v_order_book` | current order book: resting limit orders aggregated per market, side and price level | CLI `[12] Order book`, and when placing an order |
|
|---|
| 876 | | `v_order_history` | every order with its fill progress, number of trades and average fill price (from its trades) | CLI result of placing an order |
|
|---|
| 877 | | `v_trader_balances` | cash split into available and reserved, the ledger total it must equal, holdings at market value, net worth | CLI `[1] View balance` |
|
|---|
| 878 |
|
|---|
| 879 | None of these repeats a P6 report: P6 aggregates performance over a period, while these show
|
|---|
| 880 | the current state of the order book and accounts.
|
|---|
| 881 |
|
|---|
| 882 | ### Implementation
|
|---|
| 883 |
|
|---|
| 884 | #### Views
|
|---|
| 885 |
|
|---|
| 886 | ```sql
|
|---|
| 887 | CREATE VIEW project.v_active_orders AS
|
|---|
| 888 | SELECT o.id AS order_id,
|
|---|
| 889 | o.user_id,
|
|---|
| 890 | u.username,
|
|---|
| 891 | o.market_id,
|
|---|
| 892 | c.symbol,
|
|---|
| 893 | m.quote_currency,
|
|---|
| 894 | o.side,
|
|---|
| 895 | o.type,
|
|---|
| 896 | o.status,
|
|---|
| 897 | o.quantity,
|
|---|
| 898 | o.filled_quantity,
|
|---|
| 899 | o.quantity - o.filled_quantity AS remaining,
|
|---|
| 900 | o.price,
|
|---|
| 901 | CASE WHEN o.side = 'buy'
|
|---|
| 902 | THEN project.order_reservation(o.quantity - o.filled_quantity, o.price) ELSE 0 END AS reserved_cash,
|
|---|
| 903 | CASE WHEN o.side = 'sell' THEN o.quantity - o.filled_quantity ELSE 0 END AS reserved_crypto,
|
|---|
| 904 | o.placed_at
|
|---|
| 905 | FROM project.orders o
|
|---|
| 906 | JOIN project.users u ON u.id = o.user_id
|
|---|
| 907 | JOIN project.markets m ON m.id = o.market_id
|
|---|
| 908 | JOIN project.crypto c ON c.id = m.crypto_id
|
|---|
| 909 | WHERE o.status IN ('open', 'partially_filled');
|
|---|
| 910 | ```
|
|---|
| 911 |
|
|---|
| 912 | ```sql
|
|---|
| 913 | CREATE VIEW project.v_order_book AS
|
|---|
| 914 | SELECT market_id,
|
|---|
| 915 | symbol,
|
|---|
| 916 | quote_currency,
|
|---|
| 917 | side,
|
|---|
| 918 | price,
|
|---|
| 919 | SUM(remaining) AS quantity,
|
|---|
| 920 | COUNT(*) AS orders
|
|---|
| 921 | FROM project.v_active_orders
|
|---|
| 922 | WHERE type = 'limit'
|
|---|
| 923 | GROUP BY market_id, symbol, quote_currency, side, price;
|
|---|
| 924 | ```
|
|---|
| 925 |
|
|---|
| 926 | ```sql
|
|---|
| 927 | CREATE VIEW project.v_order_history AS
|
|---|
| 928 | SELECT o.id AS order_id,
|
|---|
| 929 | o.user_id,
|
|---|
| 930 | u.username,
|
|---|
| 931 | c.symbol,
|
|---|
| 932 | o.side,
|
|---|
| 933 | o.type,
|
|---|
| 934 | o.status,
|
|---|
| 935 | o.quantity,
|
|---|
| 936 | o.filled_quantity,
|
|---|
| 937 | o.quantity - o.filled_quantity AS remaining,
|
|---|
| 938 | o.price,
|
|---|
| 939 | f.trades,
|
|---|
| 940 | f.avg_fill_price,
|
|---|
| 941 | o.placed_at,
|
|---|
| 942 | o.executed_at
|
|---|
| 943 | FROM project.orders o
|
|---|
| 944 | JOIN project.users u ON u.id = o.user_id
|
|---|
| 945 | JOIN project.markets m ON m.id = o.market_id
|
|---|
| 946 | JOIN project.crypto c ON c.id = m.crypto_id
|
|---|
| 947 | LEFT JOIN LATERAL (
|
|---|
| 948 | SELECT COUNT(*) AS trades,
|
|---|
| 949 | round(SUM(t.quantity * t.price) / NULLIF(SUM(t.quantity), 0), 6) AS avg_fill_price
|
|---|
| 950 | FROM (SELECT quantity, price FROM project.market_trades WHERE buy_order_id = o.id
|
|---|
| 951 | UNION ALL
|
|---|
| 952 | SELECT quantity, price FROM project.market_trades WHERE sell_order_id = o.id) t
|
|---|
| 953 | ) f ON true;
|
|---|
| 954 | ```
|
|---|
| 955 |
|
|---|
| 956 | ```sql
|
|---|
| 957 | CREATE VIEW project.v_trader_balances AS
|
|---|
| 958 | SELECT u.id AS user_id,
|
|---|
| 959 | u.username,
|
|---|
| 960 | u.available_balance,
|
|---|
| 961 | u.reserved_balance,
|
|---|
| 962 | u.available_balance + u.reserved_balance AS total_cash,
|
|---|
| 963 | COALESCE(l.ledger_total, 0) AS ledger_total,
|
|---|
| 964 | u.invested_balance,
|
|---|
| 965 | COALESCE(p.holdings_value, 0) AS holdings_value,
|
|---|
| 966 | u.available_balance + u.reserved_balance + COALESCE(p.holdings_value, 0) AS net_worth
|
|---|
| 967 | FROM project.users u
|
|---|
| 968 | LEFT JOIN (SELECT user_id, SUM(amount) AS ledger_total
|
|---|
| 969 | FROM project.transactions GROUP BY user_id) l ON l.user_id = u.id
|
|---|
| 970 | LEFT JOIN (SELECT user_id, SUM(market_value) AS holdings_value
|
|---|
| 971 | FROM project.v_portfolio GROUP BY user_id) p ON p.user_id = u.id;
|
|---|
| 972 | ```
|
|---|
| 973 |
|
|---|
| 974 | ## Background job: filling resting limit orders
|
|---|
| 975 |
|
|---|
| 976 | ### Data requirements description
|
|---|
| 977 |
|
|---|
| 978 | **Business rule.** In EduBerza the market price is moved by the simulator (the market bot), not
|
|---|
| 979 | by users' orders. A limit order that rests in the book — a buy at or above, or a sell at or
|
|---|
| 980 | below, the current market price — must then be filled by the simulated market, at the market
|
|---|
| 981 | price, just as it would have been had the price already been there when the order was placed.
|
|---|
| 982 |
|
|---|
| 983 | **Why it is relevant, and why a background job.** Without it, a limit order could only ever
|
|---|
| 984 | fill against another user's order, and with few users most limit orders would wait forever while
|
|---|
| 985 | the market price has long passed them. That would make limit orders useless in the simulation.
|
|---|
| 986 | Nothing happens at the moment the price crosses an order that a trigger could react to: the
|
|---|
| 987 | price moves through the bot's inserts into `market_trades`, which deliberately stay cheap single
|
|---|
| 988 | inserts. Scanning and filling every crossed order on every tick inside that insert would make
|
|---|
| 989 | each tick expensive. So the work runs as a periodic job after each round of price ticks.
|
|---|
| 990 |
|
|---|
| 991 | **PostgreSQL feature.** Stored function `fill_marketable_orders()`. It is scheduled by the
|
|---|
| 992 | application, because PostgreSQL has no built-in scheduler and the faculty server provides no
|
|---|
| 993 | `pg_cron` (checked: only `plpgsql` and `pgcrypto` are available, and the project role is not a
|
|---|
| 994 | superuser).
|
|---|
| 995 |
|
|---|
| 996 | - An advisory lock (`pg_try_advisory_xact_lock`) keeps two runs from filling the same orders
|
|---|
| 997 | twice.
|
|---|
| 998 | - `FOR UPDATE … SKIP LOCKED` leaves alone an order a user is cancelling at that moment; the next
|
|---|
| 999 | run picks it up.
|
|---|
| 1000 | - Every fill goes through `execute_trade`, so all the rules above apply to it.
|
|---|
| 1001 |
|
|---|
| 1002 | **Tables affected.** `orders`, `market_trades`, `users`, `holdings`, `transactions`,
|
|---|
| 1003 | `order_events`.
|
|---|
| 1004 |
|
|---|
| 1005 | ### Implementation
|
|---|
| 1006 |
|
|---|
| 1007 | #### Stored procedures/functions
|
|---|
| 1008 |
|
|---|
| 1009 | ```sql
|
|---|
| 1010 | CREATE OR REPLACE FUNCTION project.fill_marketable_orders()
|
|---|
| 1011 | RETURNS int LANGUAGE plpgsql AS $$
|
|---|
| 1012 | DECLARE
|
|---|
| 1013 | r record;
|
|---|
| 1014 | v_count int := 0;
|
|---|
| 1015 | BEGIN
|
|---|
| 1016 | IF NOT pg_try_advisory_xact_lock(hashtext('project.fill_marketable_orders')) THEN
|
|---|
| 1017 | RETURN 0;
|
|---|
| 1018 | END IF;
|
|---|
| 1019 | FOR r IN
|
|---|
| 1020 | SELECT o.id, o.side, o.quantity - o.filled_quantity AS remaining, lp.price AS market_price
|
|---|
| 1021 | FROM project.orders o
|
|---|
| 1022 | JOIN project.markets m ON m.id = o.market_id AND m.is_active
|
|---|
| 1023 | CROSS JOIN LATERAL (SELECT project.latest_price(o.market_id) AS price) lp
|
|---|
| 1024 | WHERE o.type = 'limit'
|
|---|
| 1025 | AND o.status IN ('open', 'partially_filled')
|
|---|
| 1026 | AND (o.side = 'buy' AND o.price >= lp.price
|
|---|
| 1027 | OR o.side = 'sell' AND o.price <= lp.price)
|
|---|
| 1028 | ORDER BY o.placed_at, o.id
|
|---|
| 1029 | FOR UPDATE OF o SKIP LOCKED
|
|---|
| 1030 | LOOP
|
|---|
| 1031 | IF r.side = 'buy' THEN
|
|---|
| 1032 | PERFORM project.execute_trade(r.id, NULL, r.remaining, r.market_price, 'sell');
|
|---|
| 1033 | ELSE
|
|---|
| 1034 | PERFORM project.execute_trade(NULL, r.id, r.remaining, r.market_price, 'buy');
|
|---|
| 1035 | END IF;
|
|---|
| 1036 | v_count := v_count + 1;
|
|---|
| 1037 | END LOOP;
|
|---|
| 1038 | RETURN v_count;
|
|---|
| 1039 | END $$;
|
|---|
| 1040 | ```
|
|---|
| 1041 |
|
|---|
| 1042 | #### Scheduling
|
|---|
| 1043 |
|
|---|
| 1044 | In [`bots/main.go`](../../bots/main.go), after every round of price ticks:
|
|---|
| 1045 |
|
|---|
| 1046 | ```go
|
|---|
| 1047 | // P7 background job: the prices just moved, so fill any resting
|
|---|
| 1048 | // limit order the new market price has reached.
|
|---|
| 1049 | var filled int
|
|---|
| 1050 | if err := db.QueryRow(`SELECT fill_marketable_orders()`).Scan(&filled); err != nil {
|
|---|
| 1051 | log.Printf("fill_marketable_orders: %v", err)
|
|---|
| 1052 | } else if filled > 0 {
|
|---|
| 1053 | log.Printf(" filled %d resting limit order(s) at the new market price", filled)
|
|---|
| 1054 | }
|
|---|
| 1055 | ```
|
|---|
| 1056 |
|
|---|
| 1057 | It can also be run by hand from any SQL client: `SELECT project.fill_marketable_orders();`.
|
|---|
| 1058 |
|
|---|
| 1059 | A run with the bot, after charlie placed a limit buy of 0.1 ETH at 3599 while the market was at
|
|---|
| 1060 | 3600: the bot's random walk took the price below 3599 and the job filled the order at the market
|
|---|
| 1061 | price.
|
|---|
| 1062 |
|
|---|
| 1063 | ```
|
|---|
| 1064 | 2026/09/24 13:19:02 filled 1 resting limit order(s) at the new market price
|
|---|
| 1065 | ```
|
|---|
| 1066 |
|
|---|
| 1067 | ```
|
|---|
| 1068 | username | side | type | status | quantity | filled_quantity | price | avg_fill_price
|
|---|
| 1069 | ----------+------+-------+-----------+----------+-----------------+-------------+----------------
|
|---|
| 1070 | charlie | buy | limit | cancelled | 0.1000 | 0.0000 | 3400.000000 |
|
|---|
| 1071 | charlie | buy | limit | executed | 0.1000 | 0.1000 | 3599.000000 | 3593.979167
|
|---|
| 1072 | ```
|
|---|
| 1073 |
|
|---|
| 1074 | ## Tests proving the rules
|
|---|
| 1075 |
|
|---|
| 1076 | [`server/db/advanced_db_tests.sql`](../../server/db/advanced_db_tests.sql) plays a short trading
|
|---|
| 1077 | story on the sample data and, along the way, tries to break every rule. It starts from alice with
|
|---|
| 1078 | 8250 USD and 0.5 ETH, bob with 5000 USD, charlie with 2500 USD, and ETH/USD last traded at 3520:
|
|---|
| 1079 |
|
|---|
| 1080 | 1. alice places a limit sell of 0.3 ETH at 3600, and bob a limit buy of 0.1 at 3500. Both rest in
|
|---|
| 1081 | the book.
|
|---|
| 1082 | 2. bob places a limit buy of 0.2 at 3650. It crosses alice's ask, so they trade 0.2 at 3600:
|
|---|
| 1083 | alice's order becomes partially filled, and bob gets back the 10 he had reserved above the
|
|---|
| 1084 | trade price.
|
|---|
| 1085 | 3. charlie places a market buy of 0.15. He takes alice's remaining 0.1 from the book, and the
|
|---|
| 1086 | other 0.05 comes from the simulated market.
|
|---|
| 1087 | 4. Invalid trades, state changes and balance changes are attempted directly in SQL.
|
|---|
| 1088 | 5. bob cancels his bid.
|
|---|
| 1089 | 6. The simulator moves the price to 3450, and the background job fills bob's new limit buy at
|
|---|
| 1090 | 3500.
|
|---|
| 1091 |
|
|---|
| 1092 | The deferred checks are forced with `SET CONSTRAINTS ALL IMMEDIATE`, so a violation shows up
|
|---|
| 1093 | inside the test instead of at the final `COMMIT`. Everything is rolled back at the end. Run on
|
|---|
| 1094 | PostgreSQL 17 after `-init`:
|
|---|
| 1095 |
|
|---|
| 1096 | ```
|
|---|
| 1097 | PASS place: limit sell above the market rests in the book, crypto reserved: alice ETH reserved = 0.3000
|
|---|
| 1098 | PASS place: limit buy below the market rests in the book, cash reserved: bob available 4650.0000 reserved 350.0000
|
|---|
| 1099 | PASS event: placement recorded automatically:
|
|---|
| 1100 | PASS view: order book shows both price levels: buy 0.1000 @ 3500.000000, sell 0.3000 @ 3600.000000
|
|---|
| 1101 | PASS consistency holds after placing:
|
|---|
| 1102 | PASS place: buy without enough free cash: insufficient funds: the order needs 60000.0000, available 2500.0000
|
|---|
| 1103 | PASS place: sell more than is free (0.2 of 0.5 is already reserved): insufficient holding: trying to sell 0.3, free to sell 0.2000
|
|---|
| 1104 | PASS match: trade between the two orders at the resting price:
|
|---|
| 1105 | PASS status: seller partially filled, buyer executed (automatic): alice_ask partially_filled 0.2000/0.3000
|
|---|
| 1106 | PASS money: buyer paid 720, got back the 10 reserved above the trade price: bob available 3930.0000 reserved 350.0000
|
|---|
| 1107 | PASS crypto: 0.2 ETH moved from alice (0.1 still reserved) to bob:
|
|---|
| 1108 | PASS ledger: one buy and one sell row, linked to the orders:
|
|---|
| 1109 | PASS event: fills recorded automatically:
|
|---|
| 1110 | PASS consistency holds after the trade:
|
|---|
| 1111 | PASS market order: filled completely in two trades: executed, 2 trades, avg 3600.000000
|
|---|
| 1112 | PASS market order: alice's ask is now executed, nothing left reserved:
|
|---|
| 1113 | PASS consistency holds after the market order:
|
|---|
| 1114 | PASS trade: price above the buyer's limit: trade price 3700.000000 is outside the limit 3500.000000 of buy order …
|
|---|
| 1115 | PASS trade: more than the order has remaining: trade quantity 0.500000 exceeds the remaining quantity 0.1000 of order …
|
|---|
| 1116 | PASS trade: a sell order used as the buy side: order … is a sell order and cannot be the buy side of a trade
|
|---|
| 1117 | PASS trade: order of another market: order … is on a different market than the trade
|
|---|
| 1118 | PASS trade: executed order cannot trade again: order … is executed and cannot trade
|
|---|
| 1119 | PASS trade: a user with their own order: a user cannot trade with their own order
|
|---|
| 1120 | PASS trade: a trade that filled orders cannot be deleted: a trade that filled orders cannot be changed or deleted
|
|---|
| 1121 | PASS state: status cannot be set to executed by hand: order status executed does not match filled quantity 0.0000 of 0.1000; status is derived automatically
|
|---|
| 1122 | PASS state: filled quantity cannot be changed by hand: filled quantity of order … can only change through a trade
|
|---|
| 1123 | PASS state: executed order cannot be processed again: order … is already executed and cannot be processed again
|
|---|
| 1124 | PASS state: ordered quantity cannot change: user, market, side, type, quantity, price and placed_at of an order cannot change
|
|---|
| 1125 | PASS state: cancelling by hand without releasing the reservation: reserved balance 350.0000 does not match the 0 needed by active buy orders (user …)
|
|---|
| 1126 | PASS cancel: someone else's order: order … does not belong to this user
|
|---|
| 1127 | PASS cancel: 350 back from reserved to available, event recorded: bob available 4280.0000 reserved 0.0000
|
|---|
| 1128 | PASS cancel: a cancelled order cannot be cancelled again: order … is cancelled and cannot be cancelled
|
|---|
| 1129 | PASS trade: cancelled order cannot trade: order … is cancelled and cannot trade
|
|---|
| 1130 | PASS balance: reserving cash with no order behind it: reserved balance 100.0000 does not match the 0 needed by active buy orders (user …)
|
|---|
| 1131 | PASS balance: reserving crypto with no order behind it: reserved quantity 0.0100 does not match the 0 needed by active sell orders (user …, crypto …)
|
|---|
| 1132 | PASS balance: cash changed without a ledger row: cash 2060.0000 (available + reserved) does not match the ledger total 1960.0000 (user …)
|
|---|
| 1133 | PASS job: fills exactly the orders the new price reached: bob_bid3 executed @ 3450.000000, charlie_ask (3700) still open
|
|---|
| 1134 | PASS job: buyer paid 345, got the 5 above the fill price back:
|
|---|
| 1135 | PASS job: nothing more to do on a second run:
|
|---|
| 1136 | PASS consistency holds after the job:
|
|---|
| 1137 | PASS views: every trader's cash equals their ledger:
|
|---|
| 1138 |
|
|---|
| 1139 | passed | failed
|
|---|
| 1140 | --------+--------
|
|---|
| 1141 | 41 | 0
|
|---|
| 1142 | ```
|
|---|
| 1143 |
|
|---|
| 1144 | The same rules seen from the prototype (bob, after alice put 0.3 ETH up for sale at 3600):
|
|---|
| 1145 |
|
|---|
| 1146 | ```
|
|---|
| 1147 | -- Place buy order --
|
|---|
| 1148 | Latest price for ETH/USD = 3520.000000
|
|---|
| 1149 | Order book asks (other users' limit orders):
|
|---|
| 1150 | 3600.000000 0.3000 (1 orders)
|
|---|
| 1151 | [1] Market order (fills now at the best available price)
|
|---|
| 1152 | [2] Limit order (fills only at your price or better, otherwise waits in the order book)
|
|---|
| 1153 | > 2
|
|---|
| 1154 | Quantity: 0.2
|
|---|
| 1155 | Limit price: 3650
|
|---|
| 1156 | Order executed: buy 0.2000 ETH, average price 3600.000000
|
|---|
| 1157 | ```
|
|---|
| 1158 |
|
|---|
| 1159 | ## Changes to earlier phases
|
|---|
| 1160 |
|
|---|
| 1161 | - **Schema** ([RelationalDesign](../P2-RelationalDesign/RelationalDesign.md), [ERModel](../P1-ConceptualModel/ERModel.md)):
|
|---|
| 1162 | - `users.reserved_balance`;
|
|---|
| 1163 | - `orders.filled_quantity` and the status value `partially_filled`;
|
|---|
| 1164 | - `market_trades.buy_order_id` / `sell_order_id` (optional references to `orders` — a new
|
|---|
| 1165 | relationship "trade fills order");
|
|---|
| 1166 | - the new table `order_events`.
|
|---|
| 1167 | - **Sample data** (`data_load.sql`):
|
|---|
| 1168 | - deposit rows for bob and charlie, whose balances previously had no ledger entries behind
|
|---|
| 1169 | them;
|
|---|
| 1170 | - alice's seeded order is imported as completely filled;
|
|---|
| 1171 | - the script runs as one transaction.
|
|---|
| 1172 |
|
|---|
| 1173 | The balances documented in [BuildInstructions](../P4-Prototype/BuildInstructions.md) are
|
|---|
| 1174 | unchanged.
|
|---|
| 1175 | - **P6 demo data** (`reports_demo_data.sql`): it adjusts the users' balances by exactly what it
|
|---|
| 1176 | adds to the ledger, and imports its orders as completely filled. Both
|
|---|
| 1177 | [AdvancedReports](../P6-AdvancedReports/AdvancedReports.md) outputs are unchanged.
|
|---|
| 1178 | - **Prototype:**
|
|---|
| 1179 | - placing an order is one call to `place_order`, and market or limit can be chosen;
|
|---|
| 1180 | - new menu items `[12] Order book`, `[13] My open orders`, `[14] Cancel an order`;
|
|---|
| 1181 | - the balance screen shows reserved cash;
|
|---|
| 1182 | - the bot runs the background job.
|
|---|
| 1183 |
|
|---|
| 1184 | ## AI usage
|
|---|
| 1185 |
|
|---|
| 1186 | AI was used in this phase and is logged in full, per the course rule for P1 onward.
|
|---|
| 1187 |
|
|---|
| 1188 | - **Phase log:** [AdvancedDatabaseDevelopmentAIUsage.md](AdvancedDatabaseDevelopmentAIUsage.md)
|
|---|
| 1189 |
|
|---|
| 1190 | **In short:** the requirements — order state consistency, filled/remaining quantities, reserved
|
|---|
| 1191 | money and assets, trades only between compatible orders, the kinds of triggers, procedures and
|
|---|
| 1192 | views, and a background job only if relevant — were mine. I asked the AI to turn them into
|
|---|
| 1193 | concrete rules for the existing EduBerza database, implement them without redesigning it, test
|
|---|
| 1194 | them and document them.
|
|---|