Blame view

README.md 2.95 KB
0d8c0f816   Thanasis Naskos   initial commit
1
2
3
4
5
6
  ##### simple container just to install laravel dependencies
  
  After cloning the project, execute these commands only one time
  ```
  cd api
  cp .env-example .env
bab729aa7   Chris Bellas   Reverted UserFact...
7
  docker run --rm -v $(pwd):/app composer:2.0.7 install
0d8c0f816   Thanasis Naskos   initial commit
8
9
10
11
12
  cd ..
  ./create_volumes.sh
  docker-compose up
  docker-compose exec api php artisan migrate:fresh --seed
  ```
073260094   Thanasis Naskos   changing api to A...
13
  This first docker command just creates an empty container to install the required API dependencies.
0d8c0f816   Thanasis Naskos   initial commit
14
15
  The create_volumes script, creates the necessary folder structure and extracts the baseline elasticsearch index.
  Finally, the last docker command creates and populates the database.
25df08e6c   Chris Bellas   Merged changes fr...
16
  ## On Keycloak authentication
e228dd798   George Vlahavas   Fix typo
17
  Keycloak authentication is enabled by setting the AUTH_ENABLED variable
73196ea89   George Vlahavas   Update README wit...
18
19
20
  in the .env file to "true" (no quotes needed). Other than that, you will
  need to update the KEYCLOAK_REALM_PUBLIC_KEY value in your .env file
  too.
25df08e6c   Chris Bellas   Merged changes fr...
21

0d8c0f816   Thanasis Naskos   initial commit
22
23
24
  ##### MLTD proof of concept experiment #######
  MLTD comes with a model already trained. 
  The model is trained on the data which where available in TimescaleDB (tables XLSIEM, ADT). 
6db7cdb4f   Thanasis Naskos   Adding csv files ...
25
  The training data are provided in the csv files "xlsiem.csv" "adt.csv" for results reproduction (directory MLTD/csv_files).
0d8c0f816   Thanasis Naskos   initial commit
26
  To train a model execute the following POST request with the provided body:
0b1eae9d3   Thanasis Naskos   fixing format of ...
27
  ```
0d8c0f816   Thanasis Naskos   initial commit
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
  http://localhost:5000/api/v1.0/mltd/training
  {
  	"description":"CUREX data",
  	"timedb_host":"<the timescaleDB host>",
  	"timedb_port":5432,
  	"timedb_username":"<the timescaleDB username>",
  	"timedb_password":"<the timescaleDB password>",
  	"timedb_ssl":"False",
  	"timedb_dbname":"kea",
  	"asset_id":"server",
  	"timedb_adt_table":"adt",
  	"timedb_xlsiem_table":"xlsiem",
  	"timedb_od_table":"od",
  	"timedb_measurement":"artificial_events",
  	"mp_thres_X":10,
  	"mp_thres_Y":2,
  	"mp_thres_Z":10,
  	"mp_pat_length":6,
  	"rre":"True",
  	"rfe":"True",
  	"kofe":"False",
  	"mil_over":"True",
  	"fs":"False",
  	"rf_s":0.06,
  	"rf_midpoint":"2H",
  	"hours_before":"4H",
  	"time_segments":"20T",
  	"dates":[]
  }
0b1eae9d3   Thanasis Naskos   fixing format of ...
57
  ```
0d8c0f816   Thanasis Naskos   initial commit
58
59
60
61
62
  In order to obtain the top-k important features, use the following request:
  http://127.0.0.1:5000/api/v1.0/mltd/threat-identification/1/<k> #where 1 is the trained model id
  
  ##### OD pcap files #######
  Incide the OD directory we provide the pcap files used for load testing.
17903f0ba   Thanasis Naskos   adding OD guidelines
63
64
65
  
  To upload a pcap file for analysis use the following steps.
  First start an OD task with the following POST request and body:
0b1eae9d3   Thanasis Naskos   fixing format of ...
66
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
  http://localhost:9091/api/v1/od
  {
  	"timeDb_database": "kea",
  	"timeDb_host": "<the timescaleDB host>",
  	"timeDb_password": "<the timescaleDB password>",
  	"timeDb_port": "5432",
  	"timeDb_ssl": "true",
  	"timeDb_table": "od",
  	"timeDb_username": "postgres",
  	"k": "20",
  	"measurement": "packets-loss",
  	"mqtt_host": "localhost",
  	"mqtt_password": "",
  	"mqtt_port": "1883",
  	"mqtt_topic": "auth/od",
  	"mqtt_usermane": "",
  	"outlier_life": "0",
  	"r": "0.1",
  	"slide": "10",
  	"w": "60"
  }
0b1eae9d3   Thanasis Naskos   fixing format of ...
88
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
89
  Get the returned OD task id and execute the following POST request to upload a pcap file:
0b1eae9d3   Thanasis Naskos   fixing format of ...
90
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
91
92
93
  http://127.0.0.1:9091/api/v1/od/analyse/<OD task id>
  Header: Content-Type: application/json
  Body: file=big.pcap
0b1eae9d3   Thanasis Naskos   fixing format of ...
94
  ```
eab4a01ad   Thanasis Naskos   a test change for...
95