Key Takeaways
- Organizations are increasingly hesitant to use public AI providers due to data privacy concerns.
- Go Abacus focuses on providing AI infrastructure tailored for regulated industries.
- Banks and healthcare institutions prefer local AI solutions to maintain data privacy and control costs.
- The Go One device enables on-premises AI by connecting to employee PCs and preconfiguring software.
- The Go One device supports up to 2,000 concurrent users and can be scaled by daisy-chaining multiple devices.
- Sentry’s AI debugging agent uses comprehensive data to identify root causes and suggest fixes.
- Stress testing has significantly lowered the chances of system failure in Go Abacus’s solutions.
- The Go One OS uses specialized language models trained on client-specific data for deterministic tasks.
- Client models are trained during off-hours, with updated weights sent back nightly.
- LLMs (Large Language Models) are essentially CSV files with weights and software to run those weights.
- Go Abacus’s approach highlights the importance of privacy and cost control in AI deployment.
- The scalability and flexibility of the Go One device make it suitable for large organizations.
- Go Abacus’s solutions are designed to enhance software reliability and developer efficiency.
- The company’s client-centric approach ensures adaptability in machine learning operations.
- Understanding the structure of LLMs demystifies their operation and utility.
Guest intro
David Moscatelli is founder and CEO of Go Abacus, a company building secure, on-premises AI infrastructure for regulated industries like banking. He has achieved significant early traction with the Go1 device, a $250,000 box that enables banks to run AI models without sending data to cloud providers, and has already secured 1,600 pre-orders while hitting $1M in annual recurring revenue.
Why organizations are hesitant about public AI providers
-
Organizations are hesitant to use public AI providers due to data privacy concerns.
— David Moscatelli
- Banks, credit unions, and hospitals are wary of sending data to public AI providers.
-
They are not interested in sending their data to any of the public AI providers.
— David Moscatelli
- Concerns about data leaving their infrastructure drive the need for local AI solutions.
-
How do we use AI but locally within our network so none of our data leaves our infrastructure?
— David Moscatelli
- Privacy concerns are a significant barrier to AI adoption in sensitive sectors.
- The trend is moving towards localized AI solutions to address these privacy concerns.
- Understanding these concerns is crucial for AI deployment in regulated industries.
Go Abacus’s focus on regulated industries
-
Goabacus aims to provide AI infrastructure specifically designed for regulated industries.
— David Moscatelli
- The company targets a niche market with specific needs for privacy and compliance.
-
We’re AI infrastructure for regulated industries.
— David Moscatelli
- Understanding the challenges faced by these industries is key to Go Abacus’s business model.
- The focus on regulated industries highlights the company’s market positioning.
- Go Abacus’s solutions are tailored to meet the compliance requirements of these sectors.
- The company’s approach addresses the critical need for secure data management.
- Go Abacus is at the forefront of AI deployment in sensitive environments.
Local AI solutions for banks and healthcare
-
Banks and healthcare institutions prefer to use AI locally to maintain data privacy and control costs.
— David Moscatelli
- Local AI solutions help institutions avoid sending data to external providers.
-
How do we use AI locally within our network?
— David Moscatelli
- Cost management is a significant factor in the preference for local AI solutions.
-
You want a fixed price.
— David Moscatelli
- Understanding these preferences is crucial for AI providers targeting these sectors.
- Local solutions address both privacy and cost concerns effectively.
- The trend towards local AI solutions is growing in the banking and healthcare industries.
The Go One device and its capabilities
-
The Go One device enables on-prem AI by preconfiguring software and connecting to employee PCs.
— David Moscatelli
- The device is designed to facilitate AI access within organizations.
-
We have a piece of hardware which is the Go One.
— David Moscatelli
- The Go One device supports up to 2,000 concurrent users.
-
That device can support up to 2,000 concurrent users at one time.
— David Moscatelli
- Scalability is achieved by daisy-chaining multiple devices.
-
You can chain them together to four six eight etcetera.
— David Moscatelli
- The device’s scalability and flexibility make it suitable for large organizations.
- Understanding the technical setup of the Go One device is crucial for enterprise environments.
Enhancing software reliability with AI debugging
-
Sentry’s AI debugging agent uses comprehensive data to identify root causes of problems and suggest fixes.
— David Moscatelli
- The agent enhances software reliability and developer efficiency.
-
Sentry’s AI debugging agent uses all this data and context.
— David Moscatelli
- Efficient problem resolution is crucial in software development.
- The debugging agent’s capabilities highlight the role of AI in software maintenance.
- Understanding the importance of efficient problem resolution is key for developers.
- The agent’s use of comprehensive data ensures accurate problem identification.
- AI debugging tools are becoming essential in modern software development.
The importance of stress testing in AI systems
-
The chances of failure in our systems are incredibly low due to stress testing.
— David Moscatelli
- Stress testing ensures system robustness and reliability.
-
On the basis of our stress testing the chances of failure is incredibly low.
— David Moscatelli
- Understanding stress testing methodologies is crucial for technology providers.
- Stress testing is a critical component of system reliability.
- The effectiveness of stress testing is emphasized in Go Abacus’s solutions.
- Ensuring system robustness is essential for AI deployment in sensitive environments.
- Stress testing methodologies impact the reliability of AI systems significantly.
Specialized language models in the Go One OS
-
The Go One OS uses a collection of specialized language models that are trained on client-specific data.
— David Moscatelli
- The models perform deterministic tasks in banking and other sectors.
-
Each of those models specializes in a very specific task.
— David Moscatelli
- The use of specialized models highlights the OS’s unique architecture.
- Understanding the structure of the Go One OS is crucial for its application in banking.
- The models’ specialization ensures efficiency in performing specific tasks.
- The OS’s approach to model training is client-centric and adaptable.
- Specialized models are becoming increasingly important in AI applications.
Client-centric model training in the Go One OS
-
The company trains its models on client data during off-hours and sends the updated weights back to the clients nightly.
— David Moscatelli
- This approach ensures the models are always up-to-date with client data.
-
We batch train on their data and then we take the weights from that training.
— David Moscatelli
- The client-centric approach highlights the adaptability of the Go One OS.
- Understanding how client data is integrated into model training is crucial.
- The nightly update of weights ensures the models remain relevant and accurate.
- This approach emphasizes the importance of client-specific data in model training.
- The Go One OS’s adaptability is a key feature for its application in various sectors.
Demystifying LLMs and their structure
-
An LLM is essentially a CSV file with weights and software to run those weights.
— David Moscatelli
- This explanation simplifies the understanding of LLMs.
-
Hate to demystify for everyone that’s what it is.
— David Moscatelli
- Understanding the structure of LLMs is valuable for their operation.
- The explanation highlights the fundamental components of LLMs.
- Simplifying the concept of LLMs aids in their application and understanding.
- The demystification of LLMs is crucial for their broader adoption.
- Understanding the operation of LLMs is essential for AI practitioners.
