Blame view

README.md 2.83 KB
0d8c0f816   Thanasis Naskos   initial commit
1
2
3
4
5
6
  ##### simple container just to install laravel dependencies
  
  After cloning the project, execute these commands only one time
  ```
  cd api
  cp .env-example .env
bab729aa7   Chris Bellas   Reverted UserFact...
7
  docker run --rm -v $(pwd):/app composer:2.0.7 install
0d8c0f816   Thanasis Naskos   initial commit
8
9
10
11
12
  cd ..
  ./create_volumes.sh
  docker-compose up
  docker-compose exec api php artisan migrate:fresh --seed
  ```
073260094   Thanasis Naskos   changing api to A...
13
  This first docker command just creates an empty container to install the required API dependencies.
0d8c0f816   Thanasis Naskos   initial commit
14
15
  The create_volumes script, creates the necessary folder structure and extracts the baseline elasticsearch index.
  Finally, the last docker command creates and populates the database.
25df08e6c   Chris Bellas   Merged changes fr...
16
17
  ## On Keycloak authentication
  Update the KEYCLOAK_REALM_PUBLIC_KEY value in .env
0d8c0f816   Thanasis Naskos   initial commit
18
19
20
  ##### MLTD proof of concept experiment #######
  MLTD comes with a model already trained. 
  The model is trained on the data which where available in TimescaleDB (tables XLSIEM, ADT). 
6db7cdb4f   Thanasis Naskos   Adding csv files ...
21
  The training data are provided in the csv files "xlsiem.csv" "adt.csv" for results reproduction (directory MLTD/csv_files).
0d8c0f816   Thanasis Naskos   initial commit
22
  To train a model execute the following POST request with the provided body:
0b1eae9d3   Thanasis Naskos   fixing format of ...
23
  ```
0d8c0f816   Thanasis Naskos   initial commit
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
  http://localhost:5000/api/v1.0/mltd/training
  {
  	"description":"CUREX data",
  	"timedb_host":"<the timescaleDB host>",
  	"timedb_port":5432,
  	"timedb_username":"<the timescaleDB username>",
  	"timedb_password":"<the timescaleDB password>",
  	"timedb_ssl":"False",
  	"timedb_dbname":"kea",
  	"asset_id":"server",
  	"timedb_adt_table":"adt",
  	"timedb_xlsiem_table":"xlsiem",
  	"timedb_od_table":"od",
  	"timedb_measurement":"artificial_events",
  	"mp_thres_X":10,
  	"mp_thres_Y":2,
  	"mp_thres_Z":10,
  	"mp_pat_length":6,
  	"rre":"True",
  	"rfe":"True",
  	"kofe":"False",
  	"mil_over":"True",
  	"fs":"False",
  	"rf_s":0.06,
  	"rf_midpoint":"2H",
  	"hours_before":"4H",
  	"time_segments":"20T",
  	"dates":[]
  }
0b1eae9d3   Thanasis Naskos   fixing format of ...
53
  ```
0d8c0f816   Thanasis Naskos   initial commit
54
55
56
57
58
  In order to obtain the top-k important features, use the following request:
  http://127.0.0.1:5000/api/v1.0/mltd/threat-identification/1/<k> #where 1 is the trained model id
  
  ##### OD pcap files #######
  Incide the OD directory we provide the pcap files used for load testing.
17903f0ba   Thanasis Naskos   adding OD guidelines
59
60
61
  
  To upload a pcap file for analysis use the following steps.
  First start an OD task with the following POST request and body:
0b1eae9d3   Thanasis Naskos   fixing format of ...
62
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
  http://localhost:9091/api/v1/od
  {
  	"timeDb_database": "kea",
  	"timeDb_host": "<the timescaleDB host>",
  	"timeDb_password": "<the timescaleDB password>",
  	"timeDb_port": "5432",
  	"timeDb_ssl": "true",
  	"timeDb_table": "od",
  	"timeDb_username": "postgres",
  	"k": "20",
  	"measurement": "packets-loss",
  	"mqtt_host": "localhost",
  	"mqtt_password": "",
  	"mqtt_port": "1883",
  	"mqtt_topic": "auth/od",
  	"mqtt_usermane": "",
  	"outlier_life": "0",
  	"r": "0.1",
  	"slide": "10",
  	"w": "60"
  }
0b1eae9d3   Thanasis Naskos   fixing format of ...
84
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
85
  Get the returned OD task id and execute the following POST request to upload a pcap file:
0b1eae9d3   Thanasis Naskos   fixing format of ...
86
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
87
88
89
  http://127.0.0.1:9091/api/v1/od/analyse/<OD task id>
  Header: Content-Type: application/json
  Body: file=big.pcap
0b1eae9d3   Thanasis Naskos   fixing format of ...
90
  ```
eab4a01ad   Thanasis Naskos   a test change for...
91
92
  
  ### A test change for the integration