Blame view

README.md 2.7 KB
0d8c0f816   Thanasis Naskos   initial commit
1
2
3
4
5
6
7
8
9
10
11
12
  ##### simple container just to install laravel dependencies
  
  After cloning the project, execute these commands only one time
  ```
  cd api
  cp .env-example .env
  docker run --rm -v $(pwd):/app composer install
  cd ..
  ./create_volumes.sh
  docker-compose up
  docker-compose exec api php artisan migrate:fresh --seed
  ```
073260094   Thanasis Naskos   changing api to A...
13
  This first docker command just creates an empty container to install the required API dependencies.
0d8c0f816   Thanasis Naskos   initial commit
14
15
16
17
18
19
  The create_volumes script, creates the necessary folder structure and extracts the baseline elasticsearch index.
  Finally, the last docker command creates and populates the database.
  
  ##### MLTD proof of concept experiment #######
  MLTD comes with a model already trained. 
  The model is trained on the data which where available in TimescaleDB (tables XLSIEM, ADT). 
6db7cdb4f   Thanasis Naskos   Adding csv files ...
20
  The training data are provided in the csv files "xlsiem.csv" "adt.csv" for results reproduction (directory MLTD/csv_files).
0d8c0f816   Thanasis Naskos   initial commit
21
  To train a model execute the following POST request with the provided body:
0b1eae9d3   Thanasis Naskos   fixing format of ...
22
  ```
0d8c0f816   Thanasis Naskos   initial commit
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
  http://localhost:5000/api/v1.0/mltd/training
  {
  	"description":"CUREX data",
  	"timedb_host":"<the timescaleDB host>",
  	"timedb_port":5432,
  	"timedb_username":"<the timescaleDB username>",
  	"timedb_password":"<the timescaleDB password>",
  	"timedb_ssl":"False",
  	"timedb_dbname":"kea",
  	"asset_id":"server",
  	"timedb_adt_table":"adt",
  	"timedb_xlsiem_table":"xlsiem",
  	"timedb_od_table":"od",
  	"timedb_measurement":"artificial_events",
  	"mp_thres_X":10,
  	"mp_thres_Y":2,
  	"mp_thres_Z":10,
  	"mp_pat_length":6,
  	"rre":"True",
  	"rfe":"True",
  	"kofe":"False",
  	"mil_over":"True",
  	"fs":"False",
  	"rf_s":0.06,
  	"rf_midpoint":"2H",
  	"hours_before":"4H",
  	"time_segments":"20T",
  	"dates":[]
  }
0b1eae9d3   Thanasis Naskos   fixing format of ...
52
  ```
0d8c0f816   Thanasis Naskos   initial commit
53
54
55
56
57
  In order to obtain the top-k important features, use the following request:
  http://127.0.0.1:5000/api/v1.0/mltd/threat-identification/1/<k> #where 1 is the trained model id
  
  ##### OD pcap files #######
  Incide the OD directory we provide the pcap files used for load testing.
17903f0ba   Thanasis Naskos   adding OD guidelines
58
59
60
  
  To upload a pcap file for analysis use the following steps.
  First start an OD task with the following POST request and body:
0b1eae9d3   Thanasis Naskos   fixing format of ...
61
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
  http://localhost:9091/api/v1/od
  {
  	"timeDb_database": "kea",
  	"timeDb_host": "<the timescaleDB host>",
  	"timeDb_password": "<the timescaleDB password>",
  	"timeDb_port": "5432",
  	"timeDb_ssl": "true",
  	"timeDb_table": "od",
  	"timeDb_username": "postgres",
  	"k": "20",
  	"measurement": "packets-loss",
  	"mqtt_host": "localhost",
  	"mqtt_password": "",
  	"mqtt_port": "1883",
  	"mqtt_topic": "auth/od",
  	"mqtt_usermane": "",
  	"outlier_life": "0",
  	"r": "0.1",
  	"slide": "10",
  	"w": "60"
  }
0b1eae9d3   Thanasis Naskos   fixing format of ...
83
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
84
  Get the returned OD task id and execute the following POST request to upload a pcap file:
0b1eae9d3   Thanasis Naskos   fixing format of ...
85
  ```
17903f0ba   Thanasis Naskos   adding OD guidelines
86
87
88
  http://127.0.0.1:9091/api/v1/od/analyse/<OD task id>
  Header: Content-Type: application/json
  Body: file=big.pcap
0b1eae9d3   Thanasis Naskos   fixing format of ...
89
  ```