Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

 
Advanced search

1324814 Posts in 59741 Topics- by 50960 Members - Latest Member: marzenadobraw

December 12, 2017, 10:00:19 am

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)The turtle and the hares
Pages: [1]
Print
Author Topic: The turtle and the hares  (Read 300 times)
Whiteclaws
Level 10
*****


#include <funny.h>


View Profile
« on: June 06, 2017, 08:00:08 pm »

I just hit a wall today with some of my work. I've been brainstorming but I'd like some outsider input
I have a multi-threaded application, right:
A producer, and 3 consumers, and a massive 1M structs buffer
Producer receives message, sets message in a buffer and increments (all atomically, don't worry about race conditions)
Each consumer reads message at current position, increments by x and goes to next message, in a way that no consumer hits the same data
If a consumer doesn't find any data, then it just sleeps and waits for the producer to broadcast on the next packet (I got this working, don't worry either)
But as you can see below, and what happens in real life, is that every packet, the consumers go to sleep, because the next packet isn't made for them yet... Which can be seen below:

Example:
Consumer1:036
Consumer2:147
Consumer3:258
Producer   :012
(Notice that the first consumer goes to the next packet 2-steps before the producer gets to it's next packet

This is also the case when there's a steady inflow of packets per seconds (40000), since the consumers are SO fast, that they can handle packets at a microsecond level (It's probably gonna get slower, I didn't put encryption in yet) - but this is a problem if there's only let's say 1000 packets per second.

So my question to you, How would you make it so that producers don't sleep immediately after checking the next packet? assuming a steady flow ofc.

Thanks for reading this wall of text  Coffee
Logged
Whiteclaws
Level 10
*****


#include <funny.h>


View Profile
« Reply #1 on: June 06, 2017, 08:07:38 pm »

Note: This doesn't happen when 10/100/1000 packets are sent at the same time... without any delay (right now im dividing a second by the number of packets I want to send and sleeping that delta every time i send a packet, which shouldn't happen in real-life since packets are coming from different sources really and aren't as synchronised, I'm gonna try to create multiple clients and test on that tommorow)
I also though of giving the producer a small lead for him to fill up that buffer,
I'd love some input too
Logged
Thaumaturge
Level 6
*



View Profile WWW
« Reply #2 on: June 07, 2017, 03:22:00 pm »

Threading isn't my forte, so please take my thoughts with a grain of salt.

That said, if the consumers are processing the data faster than the producer is generating it, do you perhaps just have too many consumers? Why are there three consumers, precisely? Why not two, or even just one?
Logged


Traversal, exploration, puzzles, and combat in a heroic-fantasy setting
Website ~ Twitter
BorisTheBrave
Level 10
*****


View Profile WWW
« Reply #3 on: June 08, 2017, 11:23:08 am »

Is your diagram missing some spacing? I have have read your text but have no idea what you are trying to say.

Why would the producer ever sleep? It can keep on inserting into the queue until the queue is full, which should never happen.
Logged
Pages: [1]
Print
Jump to:  

Theme orange-lt created by panic