[D] Data labelling problems

What kind of data labelling issues do you face most often? Where do current tools fall short?

For me, I’m on a small, newly formed AI team where we have data, but we have no labelling time from SMEs.

We use Label Studio as it’s very customisable and Product have no idea what they want yet. It’s self hosted as our data is highly sensitive.

I already have some gripes about Label Studio:

• Poor search for high-cardinality categorical labels

• Review, role management etc. limited to the Enterprise plan

• No ability to hide existing labels from additional labellers to avoid anchoring bias

• I could go on

Curious to hear others’ experiences.