Allow choosing an allocation strategy in controllers #31

New issue

Closed

opened 2022-10-15 07:17:01 +00:00 by heartles · 1 comment

heartles commented

2022-10-15 07:17:01 +00:00

Owner

Instead, we should have the allocation strategy decided by the endpoint, and the controller helper then is responsible for creating an arena allocator as necessary. The API connection then just uses the allocator it is passed in.

I expect that most calls will be best served by the arena allocator, but doing so by default may be confusing, so arena-based allocation should be opt-in.

Right now, ApiConn is hard coded to use an arena allocator and base controller data is allocated using the base allocator. This works fine for short-lived endpoints. However, in longer lasting endpoints like a streaming endpoint, the arena allocator may cause the endpoint to slowly consume more and more memory. Instead, we should have the allocation strategy decided by the endpoint, and the controller helper then is responsible for creating an arena allocator as necessary. The API connection then just uses the allocator it is passed in. I expect that most calls will be best served by the arena allocator, but doing so by default may be confusing, so arena-based allocation should be opt-in.