Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Should GenAI companies be audited for their use of customer data?
9 points by darshitac_ 3 days ago | hide | past | favorite | 6 comments
# Background

The GenAI industry is facing a significant challenge: widespread mistrust between buyers and their GenAI vendors. There is deep-seated fear that vendors will exploit their buyer’s proprietary data to train or fine-tune their models.

In many cases, the actions of established companies have perpetuated this fear. For example, Slack's privacy policy update granted it permission to use customer data to train its AI models without requiring explicit consent.

Several factors contribute to this mistrust:

1. Vague descriptions of how customer data is used in the Terms of Service, which may give vendors undue leeway 2. Concerns that established terms will not be fully respected 3. Terms that are often changed without prior notice

As a result of these concerns, buyers are taking various approaches to protect their intellectual property when making buying decisions for GenAI products:

1. Most don't want to buy from GenAI companies at all 2. Many require on-prem deployment 3. Many insist on strict contractual provisions against using their data for training or fine-tuning 4. Many have developed internal practices to contextualize data before sending it to any AI company

# Problem

The current situation appears to be a lose-lose for both buyers and vendors. Buyers are unable to take advantage of potentially transformative AI technology, while vendors struggle to close deals due to a lack of trust.

# Solution

One potential solution is to require independent audits of GenAI systems. Each audit would focus on verifying compliance with specific contractual requirements, such as:

1. Ensuring customer data is not used for fine-tuning 2. Confirming customer data is deleted after its intended use

# Ask

If you are responsible for making buying decisions in your company, will an independent audit report that verifies compliance with contractual requirements assuage your IP-related concerns?






I would say any company with user data should need to be audited to ensure it is kept private and not misused. I wouldn’t limit it to GenAI companies.

I guess we need divide and conquer the data mis-use problem. At this point, we are not sure what will be a good starting point.

Let's start with social media, which has everyone's names, photos, locations, locations from geotagged photos, contacts, maps relationship strength between contacts, sleep and work patterns, etc.

I agree - AI model training is just one way of misusing customer data.

Is this GenAI-generated?

Haha, 100% human generated.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: