How memory works and how programmers can utilize it better?
In these memory articles, we will discuss more about different types of memory and how they can be utilized better by programmers.
Pooh: So sunny, when I started in Embedded systems, I worked with simple memory where there was no cache, no concept of static and dynamic RAM. What do you think is the need for complicated memory. Why can’t we have simple memory devices as is the case in small microcontrollers.
Sunny: Well Pooh, because of commercial reasons and technical limitations, most of the research in computer field has been on increasing the speed of processors, because of which memory devices’ speed fell behind and this resulted in bottlenecks. For cost reasons now, mass storage devices and memory subsystems have fallen behind in speed a lot as compared to processors and hence to remove the bottlenecks, we have different hardware techniques which makes memory complicated.
Pooh: So Sunny, to make it clear, when you say mass storage devices you mean devices such as Flash drives, hard disk drives etc. But when you are saying memory, you mostly mean the RAM memory.
Sunny: Yhhh Pooh, in fact mass storage devices are too slow and that’s why operating systems make a cache of most used data from the mass storage memory in the main memory. And its quite simple to use.
Pooh: Then let’s talk about main memory then. Why is it more complicated and not as simple as mass storage devices. And again when you say cache of mass storage memory, it sounds like you are talking about making a copy of the data from mass storage memory into main memory.
Sunny: That’s right, caches are nothing but copies of data from a particular memory into another memory which is faster to use. The reason why main memory optimization techniques are complicated is a sum of different reasons. We can list them as
- RAM hardware design which is mostly focused on speed as well as parallelism
- Memory controller design
- CPU caches
- DMA (Direct memory access)
Now each of these hardware design techniques makes accessing the main memory more complicated as compared to the mass storage devices.
Pooh: Let’s talk about these memories first and then we can deep dive into the hardware techniques later. Now let me take the center stage and tell you my idea of memory. And as per your terminology, when I say memory, I mean the main memory or let’s say the RAM memory.
Sunny: Sure, please go ahead.
Pooh: So, as we see it, most of the computers or data centers these days are generic, and we can call them generic hardware. Data centers have multiple sockets, and each socket has its own CPU or CPUs. In the initial days, our personal computers generally have a chipset with two parts: the Northbridge and the Southbridge.
This is the simplest structure of a CPU where we have only two CPUs. Both these CPUs are connected to the Northbridge via a common bus (Front Side Bus). The NorthBridge can be thought of as a memory controller which is connected to the main memory. Based on the memory controller, different types of RAM such as static RAM, DRAM etc. can be used. To communicate with any other device the Northbridge has to communicate with the SouthBridge. The Southbridge since connected to all other system devices is also known as I/O bridge. This system has particular characteristics
To communicate with each other, the CPU uses the same bus which is shared with the Northbridge.
- All communication with RAM must pass through the Northbridge
- The RAM has only a single port.
- To communicate with any I/O device, the CPU has to first go through the Northbridge.
Now can you put some light on the drawbacks or bottlenecks of such system.
Sunny: The first thing is that if devices want to communicate with RAM, they have to use CPU for it. This bottleneck was resolved by introducing DMA where the device could directly access RAM through the Northbridge without the use of CPU. But introducing DMA gives rise to another problem, DMA requests and CPU requests have to compete with each other for RAM contention. The second bottleneck was the bus design between Northbridge and RAM. Earlier it was a single bus to RAM chips, so parallelism was not possible. But then came DDR2 which used two different buses for communicating with RAM. Similarly, DDR3, DDR4 and DDR5 increased the bandwidth even more.
Pooh: That makes sense. I will hold you here, to introduce the concept of DMA in single core microcontroller based systems or DSP systems where the Northbridge and Southbridge terminology don’t exist. But the Northbridge is replaced by a memory controller and the Southbridge is replaced by a peripheral controller.
Sunny: That’s a great analogy and an accurate one. The memory controller is the part that is connected to main memory and DMA allows us to bypass the core and use the main memory directly without using the CPU. Let’s keep the discussion flowing about the evolution of the system and their memories. I need to take a break from this so let’s meet again soon for part 2 of this.
Leave a Reply