Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

Advanced search

1301551 Posts in 58000 Topics- by 49066 Members - Latest Member: verborghs

June 23, 2017, 02:28:05 am

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)The turtle and the hares
Pages: [1]
Author Topic: The turtle and the hares  (Read 153 times)
Level 10

#include <funny.h>

View Profile Email
« on: June 06, 2017, 08:00:08 pm »

I just hit a wall today with some of my work. I've been brainstorming but I'd like some outsider input
I have a multi-threaded application, right:
A producer, and 3 consumers, and a massive 1M structs buffer
Producer receives message, sets message in a buffer and increments (all atomically, don't worry about race conditions)
Each consumer reads message at current position, increments by x and goes to next message, in a way that no consumer hits the same data
If a consumer doesn't find any data, then it just sleeps and waits for the producer to broadcast on the next packet (I got this working, don't worry either)
But as you can see below, and what happens in real life, is that every packet, the consumers go to sleep, because the next packet isn't made for them yet... Which can be seen below:

Producer   :012
(Notice that the first consumer goes to the next packet 2-steps before the producer gets to it's next packet

This is also the case when there's a steady inflow of packets per seconds (40000), since the consumers are SO fast, that they can handle packets at a microsecond level (It's probably gonna get slower, I didn't put encryption in yet) - but this is a problem if there's only let's say 1000 packets per second.

So my question to you, How would you make it so that producers don't sleep immediately after checking the next packet? assuming a steady flow ofc.

Thanks for reading this wall of text  Coffee
Level 10

#include <funny.h>

View Profile Email
« Reply #1 on: June 06, 2017, 08:07:38 pm »

Note: This doesn't happen when 10/100/1000 packets are sent at the same time... without any delay (right now im dividing a second by the number of packets I want to send and sleeping that delta every time i send a packet, which shouldn't happen in real-life since packets are coming from different sources really and aren't as synchronised, I'm gonna try to create multiple clients and test on that tommorow)
I also though of giving the producer a small lead for him to fill up that buffer,
I'd love some input too
Level 4

View Profile WWW Email
« Reply #2 on: June 07, 2017, 03:22:00 pm »

Threading isn't my forte, so please take my thoughts with a grain of salt.

That said, if the consumers are processing the data faster than the producer is generating it, do you perhaps just have too many consumers? Why are there three consumers, precisely? Why not two, or even just one?

Traversal, exploration, puzzles, and combat in a heroic-fantasy setting
Website ~ Twitter
Level 10

View Profile WWW
« Reply #3 on: June 08, 2017, 11:23:08 am »

Is your diagram missing some spacing? I have have read your text but have no idea what you are trying to say.

Why would the producer ever sleep? It can keep on inserting into the queue until the queue is full, which should never happen.
Pages: [1]
Jump to:  

Theme orange-lt created by panic