Machine Learning Driven Web Log Tracing system
Project Description
Use behavior analysis is nowadays the challenge for IT management on how to stop network security incident. The key is to determine if the user account is legitimate or credential is already been hacked.
Many of the corporations considered that implementing of log management system is equivalent to the use of big-data platform for log management and analysis. However, that is just a mis-understanding of true big-data log management platform.
Besides, collection of web logs cannot immediately generate alerts and are rather difficult to identify whether logs are generate from human, bots or attacker.
Machine learning is specialised to be used for determining patterns of user behaviour and web logs can be used for tracing & tracking user activities determine the baseline and profiling of user patterns.
Many of the corporations considered that implementing of log management system is equivalent to the use of big-data platform for log management and analysis. However, that is just a mis-understanding of true big-data log management platform.
Besides, collection of web logs cannot immediately generate alerts and are rather difficult to identify whether logs are generate from human, bots or attacker.
Machine learning is specialised to be used for determining patterns of user behaviour and web logs can be used for tracing & tracking user activities determine the baseline and profiling of user patterns.
Supervisor
IEONG Sze Chung Ricci
Co-Supervisor
IEONG Sze Chung Ricci
Quota
2
Course type
UROP1100
UROP2100
UROP3100
Applicant's Roles
In this research, applicants will have to handle the following:
- Setup and implement the log processing in ElasticSearch, Logstash and Kibana
- Analyse the web logs and define the data model for web log analysis
- Define mechanism for determining whether logs are generated by bots
- Define the web log distribution pattern and detect abnormality based on pattern as well as risk score evaluation
- Setup and implement the log processing in ElasticSearch, Logstash and Kibana
- Analyse the web logs and define the data model for web log analysis
- Define mechanism for determining whether logs are generated by bots
- Define the web log distribution pattern and detect abnormality based on pattern as well as risk score evaluation
Applicant's Learning Objectives
- To use current open source big-data log analysis platform
- To perform security log data analytics
- To identify web log distribution pattern
- To understand mechanism of abnormal attack patterns
- To perform security log data analytics
- To identify web log distribution pattern
- To understand mechanism of abnormal attack patterns
Complexity of the project
Challenging