Kx.com | KXCON23 | High Performance, Real-time Event Processing with PyKX | kdb at Citadel

Citadel has an effective framework for high-performance real-time event processing using Kdb+ and several of the firm’s platforms have Kdb+ either at the core or close to it. Citadel’s framework builds on the Kdb+ primitives to allow engineers to rapidly develop dynamic event processing systems, from which insights can be drawn in real-time. Despite its advantages, the Q language presents a strong barrier to entry for those unfamiliar with Q. That’s why we’re thrilled by the introduction of PyKx. In this talk, we’ll be taking you through our journey in integrating it into our framework and the challenges it’s helped overcome.

PyKX is a Pythonic interface to kdb+, a high-performance time-series database and analytics engine. PyKX makes it easy to develop and deploy high-performance, real-time event processing applications using Python.

PyKX provides a number of features that make it well-suited for real-time event processing, including:

• High performance: PyKX is based on kdb+, which is one of the fastest time-series databases and analytics engines available.
• Scalability: PyKX is able to scale to handle very large volumes of data and transactions.
• Ease of use: PyKX provides a Pythonic interface to kdb+, which makes it easy for Python developers to develop and deploy real-time event processing applications.

PyKX can be used to develop a wide variety of real-time event processing applications, such as:

• Fraud detection: PyKX can be used to develop real-time fraud detection systems that can identify and flag fraudulent transactions in real time.
• Risk management: PyKX can be used to develop real-time risk management systems that can identify and assess risks in real time.
• Market surveillance: PyKX can be used to develop real-time market surveillance systems that can monitor market activity for unusual or suspicious behavior.
• Trading: PyKX can be used to develop real-time trading systems that can execute trades in real time.

Here are some examples of companies that are using PyKX for real-time event processing:

• Citadel
• Goldman Sachs
• JPMorgan Chase
• Nasdaq
• Thomson Reuters

These companies are using PyKX to develop a wide variety of real-time event processing applications, such as fraud detection, risk management, market surveillance, and trading.

PyKX is a powerful tool for developing high-performance, real-time event processing applications. PyKX is easy to use and can be scaled to handle very large volumes of data and transactions. PyKX is used by some of the world’s leading financial institutions and other companies to develop real-time event processing applications for a variety of purposes.

Here are some tips for developing high-performance, real-time event processing applications with PyKX:

• Use vectorized processing whenever possible. Vectorized processing allows PyKX to perform operations on multiple data elements simultaneously, which can significantly improve performance.
• Use parallel processing whenever possible. Parallel processing allows PyKX to distribute its workload across multiple processors, which can improve scalability.
• Use a cache to store frequently accessed data. This can reduce the number of database queries and improve performance.
• Use a high-performance messaging system to communicate with other systems. This can help to ensure that your application is able to keep up with the flow of data.
• Monitor your application’s performance and make adjustments as needed.

By following these tips, you can develop high-performance, real-time event processing applications with PyKX