Knee or ROC

Self-attention transformers have demonstrated accuracy for image classification with smaller data sets. However, a limitation is that tests to-date are based upon single class image detection with known representation of image populations. For instances where the input image classes may be greater than one and test sets that lack full information on representation of image populations, accuracy calculations must adapt. The Receiver Operating Characteristic (ROC) accuracy thresh-old can address the instances of multi-class input images. However, this approach is unsuitable in instances where image population representation is unknown. We consider calculating accuracy using the knee method to determine threshold values on an ad-hoc basis. Results of ROC curve and knee thresholds for a multi-class data set, created from CIFAR-10 images, are discussed for multi-class image detection.

Personality of AI

This research paper delves into the evolving landscape of fine-tuning large language models (LLMs) to align with human users, extending beyond basic alignment to propose "personality alignment" for language models in organizational settings. Acknowledging the impact of training methods on the formation of undefined personality traits in AI models, the study draws parallels with human fitting processes using personality tests. Through an original case study, we demonstrate the necessity of personality fine-tuning for AIs and raise intriguing questions about applying human-designed tests to AIs, engineering specialized AI personality tests, and shaping AI personalities to suit organizational roles. The paper serves as a starting point for discussions and developments in the burgeoning field of AI personality alignment, offering a foundational anchor for future exploration in human-machine teaming and co-existence.

Using a Neural Network to Detect Anomalies given an N-gram Profile

In order to detect unknown intrusions and runtime errors of computer programs, the cyber-security community has developed various detection techniques. Anomaly detection is an approach that is designed to profile the normal runtime behavior of computer programs in order to detect intrusions and errors as anomalous deviations from the observed normal. However, normal but unobserved behavior can trigger false positives. This limitation has significantly decreased the practical viability of anomaly detection techniques. Reported approaches to this limitation span a simple alert threshold definition to distribution models for approximating all normal behavior based on the limited observation. However, each assumption or approximation poses the potential for even greater false positive rates. This paper presents our study on how to explain the presence of anomalies using a neural network, particularly Long Short-Term Memory, independent of actual data distributions. We present and compare three anomaly detection models, and report on our experience running different types of attacks on an Apache Hypertext Transfer Protocol server. We performed a comparative study, focusing on each model's ability to detect the onset of each attack while avoiding false positives resulting from unknown normal behavior. Our best-performing model detected the true onset of every attack with zero false positives.

On Analyzing Churn Prediction in Mobile Games

In subscription-based businesses, the churn rate refers to the percentage of customers who discontinue their subscriptions within a given time period. Particularly, in the mobile games industry, the churn rate is often pronounced due to the high competition and cost in customer acquisition; therefore, the process of minimizing the churn rate is crucial. This needs churn prediction, predicting users who will be churning within a given time period. Accurate churn prediction can enable the businesses to devise and engage strategic remediations to maintain a low churn rate. The paper presents our highly accurate churn prediction method. We designed this method to take into account each individual user's distinct usage period in churn prediction. As presented in the paper, this approach was able to achieve 96.6% churn prediction accuracy on a real game business. In addition, the paper shows that other existing churn prediction algorithms are improved in prediction accuracy when this method is applied.