Streaming API doesn't use chunked encoding (breaks bitlbee-mastodon) #416

Open
opened 2018-12-05 15:31:58 +00:00 by zimpenfish · 11 comments
Member

bitlbee-mastodon tries to use the streaming API (/api/v1/streaming/user) but gets confused due to the connection being a "normal" HTTP connection (Content-Length, keep-alive) rather than being chunked (no Content-Length, Transfer-Encoding: chunked). The underlying HTTP code in bitlbee records a closed stream when req->body_size >= req->content_length which causes bitlbee-mastodon to consider login failed (Login error: Stream closed (200 OK)).

Mastodon Streaming API docs says that the streaming endpoints work as "chunked-encoding transfer" (or, alternately, a websocket).

(bitlbee-mastodon is A-OK talking to a Mastodon instance; it's just Pleroma that confuses it.)

[bitlbee-mastodon](https://alexschroeder.ch/software/Bitlbee_Mastodon) tries to use the streaming API (`/api/v1/streaming/user`) but gets confused due to the connection being a "normal" HTTP connection (`Content-Length`, `keep-alive`) rather than being chunked (no `Content-Length`, `Transfer-Encoding: chunked`). The underlying HTTP code in bitlbee records a closed stream when `req->body_size >= req->content_length` which causes bitlbee-mastodon to consider login failed (`Login error: Stream closed (200 OK)`). [Mastodon Streaming API docs](https://docs.joinmastodon.org/api/streaming/) says that the streaming endpoints work as "chunked-encoding transfer" (or, alternately, a websocket). (bitlbee-mastodon is A-OK talking to a Mastodon instance; it's just Pleroma that confuses it.)
Member

hi @href, could you help find pont of problem.
I don't sure that problem in chunked encoding. I see that bitlbee-mastodon used /api/v1/streaming/user as stream API endpoint, but Pleroma use like this "/api/v1/streaming?access_token=au..g=&stream=user" endpoint.

hi @href, could you help find pont of problem. I don't sure that problem in chunked encoding. I see that bitlbee-mastodon used /api/v1/streaming/user as stream API endpoint, but Pleroma use like this "/api/v1/streaming?access_token=au..g=&stream=user" endpoint.
Member

@parallel588 as Streaming API Mastodon docs specify GET /api/v1/streaming/user,GET /api/v1/streaming/public etc... are used for HTTP events, while /api/v1/streaming is used for websockets

@parallel588 as [Streaming API Mastodon docs](https://source.joinmastodon.org/mastodon/docs/blob/master/content/en/api/streaming.md) specify `GET /api/v1/streaming/user`,`GET /api/v1/streaming/public` etc... are used for HTTP events, while `/api/v1/streaming` is used for websockets
Member

@rinpatch , yes. you are right. but I cannot find similar GET /api/v1/streaming/user (GET /api/v1/streaming/public, ..) routes in Pleroma.
i see redirect for any GET /api/v1/streaming/...

[info] Running Pleroma.Web.Endpoint with cowboy 1.1.2 at http://localhost:4000 [info] Gopher server disabled [info] GET /api/v1/streaming/user [debug] Processing with Fallback.RedirectController.redirector/2 Parameters: %{"path" => ["api", "v1", "streaming", "user"]} Pipelines: []

@rinpatch , yes. you are right. but I cannot find similar GET /api/v1/streaming/user (GET /api/v1/streaming/public, ..) routes in Pleroma. i see redirect for any GET /api/v1/streaming/... ` [info] Running Pleroma.Web.Endpoint with cowboy 1.1.2 at http://localhost:4000 [info] Gopher server disabled [info] GET /api/v1/streaming/user [debug] Processing with Fallback.RedirectController.redirector/2 Parameters: %{"path" => ["api", "v1", "streaming", "user"]} Pipelines: [] `
Member

maybe I just don't understand where to look for the problem :)

maybe I just don't understand where to look for the problem :)
Member

Took a look at the source. Can't find it either, maybe it is not even supported?

Took a look at the source. Can't find it either, maybe it is not even supported?

Yes, they aren't supported (yet). This would be about implementing them.

Yes, they aren't supported (yet). This would be about implementing them.
Member

great, we found problem. :)
then I'll add pleroma/web/streaming_api where we place controller streaming_api_controller.ex and context streaming_api.ex or we need to other namespace (maybe mastodon_api)

great, we found problem. :) then I'll add pleroma/web/streaming_api where we place controller streaming_api_controller.ex and context streaming_api.ex or we need to other namespace (maybe mastodon_api)
Member

@parallel588 i'll take care of fixing this. Got some (a lot) of spare time on a train tomorrow; and already dealt with SSE in Cowoboy

@parallel588 i'll take care of fixing this. Got some (a lot) of spare time on a train tomorrow; and already dealt with SSE in Cowoboy
Member

ok

ok

@href how's it looking?

@href how's it looking?
Owner

bumping this again

bumping this again
Sign in to join this conversation.
No milestone
No project
No assignees
6 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
pleroma/pleroma#416
No description provided.