Preventing Abuse Using Unsupervised Learning

Описание: Detection of abusive activity on a large social network is an adversarial challenge with quickly evolving behavior patterns and imperfect ground truth labels. Additionally, there is often negligible signal from an individual fake account before it is used for abuse. These characteristics limit the use of supervised learning techniques, but they can be overcome using unsupervised methods. To address these challenges, we created a Scala/Spark implementation of the isolation forest unsupervised outlier detection algorithm using the Spark ML API; we recently open sourced this library (https://github.com/linkedin/isolation.... We also developed a Scala/Spark unsupervised fuzzy clustering library to identify groups of abusive accounts with similar behavior. Key takeaways: In this talk, we describe the details of these unsupervised algorithms, why this type of machine learning is well suited to the specific challenges associated with fighting abuse, and how we use these models to detect accounts (fake, compromised, or real) that are engaging in abusive behavior such as automation and fake engagement at LinkedIn. We will also discuss best practices and lessons learned from our experience using these techniques in production.

About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unifie...
Connect with us:
Website: https://databricks.com
Facebook:   / databricksinc  
Twitter:   / databricks  
LinkedIn:   / databricks  
Instagram:   / databricksinc   Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-nam...