Q Language - Table


Advertisements

Tables are at the heart of kdb+. A table is a collection of named columns implemented as a dictionary. q tables are column-oriented.

Creating Tables

Tables are created using the following syntax −

q)trade:([]time:();sym:();price:();size:())

q)trade
time sym price size
-------------------

In the above example, we have not specified the type of each column. This will be set by the first insert into the table.

Another way, we can specify column type on initialization −

q)trade:([]time:`time$();sym:`$();price:`float$();size:`int$())

Or we can also define non-empty tables −

q)trade:([]sym:(`a`b);price:(1 2))

q)trade

 sym   price
-------------
  a      1
  b      2

If there are no columns within the square brackets as in the examples above, the table is unkeyed.

To create a keyed table, we insert the column(s) for the key in the square brackets.

q)trade:([sym:`$()]time:`time$();price:`float$();size:`int$())

q)trade

 sym   | time price size
-----  | ---------------

One can also define the column types by setting the values to be null lists of various types −

q)trade:([]time:0#0Nt;sym:0#`;price:0#0n;size:0#0N)

Getting Table Information

Let’s create a trade table −

trade: ([]sym:`ibm`msft`apple`samsung;mcap:2000 4000 9000 6000;ex:`nasdaq`nasdaq`DAX`Dow)

q)cols trade                         / column names of a table
`sym`mcap`ex

q)trade.sym                          / Retrieves the value of column sym
`ibm`msft`apple`samsung

q)show meta trade                    / Get the meta data of a table trade.

  c   | t f a
----- | -----
 Sym  | s
 Mcap | j
 ex   | s

Primary Keys and Keyed Tables

Keyed Table

A keyed table is a dictionary that maps each row in a table of unique keys to a corresponding row in a table of values. Let us take an example −

val:flip `name`id!(`John`Jenny`Jonathan;9 18 27)
                          / a flip dictionary create table val
id:flip (enlist `eid)!enlist 99 198 297
                          / flip dictionary, having single column eid

Now create a simple keyed table containing eid as key,

q)valid: id ! val

q)valid                 / table name valid, having key as eid

  eid |  name      id
------| ---------------
  99  |  John      9
  198 |  Jenny     18
  297 |  Jonathan  27

ForeignKeys

A foreign key defines a mapping from the rows of the table in which it is defined to the rows of the table with the corresponding primary key.

Foreign keys provide referential integrity. In other words, an attempt to insert a foreign key value that is not in the primary key will fail.

Consider the following examples. In the first example, we will define a foreign key explicitly on initialization. In the second example, we will use foreign key chasing which does not assume any prior relationship between the two tables.

Example 1 − Define foreign key on initialization

q)sector:([sym:`SAMSUNG`HSBC`JPMC`APPLE]ex:`N`CME`DAQ`N;MC:1000 2000 3000 4000)

q)tab:([]sym:`sector$`HSBC`APPLE`APPLE`APPLE`HSBC`JPMC;price:6?9f)

q)show meta tab

  c    | t f a
------ | ----------
 sym   | s sector
 price | f

q)show select from tab where sym.ex=`N

  sym     price
----------------
 APPLE   4.65382
 APPLE   4.643817
 APPLE   3.659978

Example 2 − no pre-defined relationship between tables

sector: ([symb:`IBM`MSFT`HSBC]ex:`N`CME`N;MC:1000 2000 3000)
tab:([]sym:`IBM`MSFT`MSFT`HSBC`HSBC;price:5?9f)

To use foreign key chasing, we must create a table to key into sector.

q)show update mc:(sector([]symb:sym))[`MC] from tab

  sym     price      mc
--------------------------
  IBM   7.065297    1000
  MSFT  4.812387    2000
  MSFT  6.400545    2000
  HSBC  3.704373    3000
  HSBC  4.438651    3000

General notation for a predefined foreign key −

select a.b from c where a is the foreign key (sym), b is a

field in the primary key table (ind), c is the

foreign key table (trade)

Manipulating Tables

Let’s create one trade table and check the result of different table expression −

q)trade:([]sym:5?`ibm`msft`hsbc`samsung;price:5?(303.00*3+1);size:5?(900*5);time:5?(.z.T-365))

q)trade

  sym        price   size   time
-----------------------------------------
 msft      743.8592  3162  02:32:17.036
 msft      641.7307  2917  01:44:56.936
 hsbc      838.2311  1492  00:25:23.210
 samsung   278.3498  1983  00:29:38.945
 ibm       838.6471  4006  07:24:26.842

Let us now take a look at the statements that are used to manipulate tables using q language.

Select

The syntax to use a Select statement is as follows −

select [columns] [by columns] from table [where clause]

Let us now take an example to demonstrate how to use Select statement −

q)/ select expression example

q)select sym,price,size by time from trade where size > 2000

    time      |  sym    price     size
------------- | -----------------------
 01:44:56.936 |  msft   641.7307  2917
 02:32:17.036 |  msft   743.8592  3162
 07:24:26.842 |  ibm    838.6471  4006

Insert

The syntax to use an Insert statement is as follows −

`tablename insert (values)
Insert[`tablename; values]

Let us now take an example to demonstrate how to use Insert statement −

q)/ Insert expression example

q)`trade insert (`hsbc`apple;302.0 730.40;3020 3012;09:30:17.00409:15:00.000)
5 6

q)trade

   sym    price     size    time
------------------------------------------
  msft    743.8592  3162   02:32:17.036
  msft    641.7307  2917   01:44:56.936
  hsbc    838.2311  1492   00:25:23.210
 samsung  278.3498  1983   00:29:38.945
  ibm     838.6471  4006   07:24:26.842
  hsbc    302       3020   09:30:17.004
  apple   730.4     3012   09:15:00.000

q)/Insert another value

q)insert[`trade;(`samsung;302.0; 3333;10:30:00.000]
']

q)insert[`trade;(`samsung;302.0; 3333;10:30:00.000)]
,7

q)trade

   sym     price   size     time
----------------------------------------
  msft   743.8592  3162  02:32:17.036
  msft   641.7307  2917  01:44:56.936
  hsbc   838.2311  1492  00:25:23.210
 samsung 278.3498  1983  00:29:38.945
  ibm    838.6471  4006  07:24:26.842
  hsbc   302       3020  09:30:17.004
  apple  730.4     3012  09:15:00.000
 samsung 302       3333  10:30:00.000

Delete

The syntax to use a Delete statement is as follows −

delete columns from table
delete from table where clause

Let us now take an example to demonstrate how to use Delete statement −

q)/Delete expression example

q)delete price from trade

   sym   size       time
-------------------------------
  msft   3162   02:32:17.036
  msft   2917   01:44:56.936
  hsbc   1492   00:25:23.210
 samsung 1983   00:29:38.945
  ibm    4006   07:24:26.842
  hsbc   3020   09:30:17.004
  apple  3012   09:15:00.000
 samsung 3333   10:30:00.000

q)delete from trade where price > 3000

   sym     price     size       time
-------------------------------------------
  msft    743.8592   3162    02:32:17.036
  msft    641.7307   2917    01:44:56.936
  hsbc    838.2311   1492    00:25:23.210
 samsung  278.3498   1983    00:29:38.945
  ibm     838.6471   4006    07:24:26.842
  hsbc    302        3020    09:30:17.004
  apple   730.4      3012    09:15:00.000
 samsung  302        3333    10:30:00.000

q)delete from trade where price > 500

  sym     price     size     time
-----------------------------------------
 samsung  278.3498  1983  00:29:38.945
  hsbc    302       3020  09:30:17.004
 samsung  302       3333  10:30:00.000

Update

The syntax to use an Update statement is as follows −

update column: newValue from table where ….

Use the following syntax to update the format/datatype of a column using the cast function −

update column:newValue from `table where …

Let us now take an example to demonstrate how to use Update statement −

q)/Update expression example

q)update size:9000 from trade where price > 600

  sym     price      size     time
------------------------------------------
  msft    743.8592   9000   02:32:17.036
  msft    641.7307   9000   01:44:56.936
  hsbc    838.2311   9000   00:25:23.210
 samsung  278.3498   1983   00:29:38.945
  ibm     838.6471   9000   07:24:26.842
  hsbc    302        3020   09:30:17.004
  apple   730.4      9000   09:15:00.000
 samsung  302        3333   10:30:00.000

q)/Update the datatype of a column using the cast function

q)meta trade

   c  |  t f a
----- | --------
  sym |  s
 price|  f
 size |  j
 time |  t

q)update size:`float$size from trade

  sym     price     size      time
------------------------------------------
  msft    743.8592  3162    02:32:17.036
  msft    641.7307  2917    01:44:56.936 
  hsbc    838.2311  1492    00:25:23.210
 samsung  278.3498  1983    00:29:38.945
  ibm     838.6471  4006    07:24:26.842
  hsbc    302       3020    09:30:17.004
  apple   730.4     3012    09:15:00.000
 samsung  302       3333    10:30:00.000

q)/ Above statement will not update the size column datatype permanently

q)meta trade

   c   |  t f a
------ | --------
  sym  |   s
 price |   f
 size  |   j
 time  |   t
 
q)/to make changes in the trade table permanently, we have do

q)update size:`float$size from `trade
`trade

q)meta trade

   c   |  t f a
------ | --------
  sym  |   s
 price |   f
 size  |   f
 time  |   t
Advertisements