After understanding cache terminology, it is important to understand the effects that a cache has on access time and traffic to the lower level.

The following algorithm is used by a cache controller in order to take advantage of both temporal and spatial locality.

    if the requested data is in the local store // cache hit, cache access time
        return the requested data
    else // cache miss, lower-level access time
        get the requested and nearby data from the lower level
        save it in the local store
        return the requested data

The fraction or percentage of accesses that result in a hit is called the hit rate.

The fraction or percentage of accesses that result in a miss is called the miss rate.

It follows that hit rate + miss rate = 1.0 (100%).

The difference between lower level access time and cache access time is called the miss penalty.

Effective access time is a standard effective average.

    effective-access-time = hit-rate * cache-access-time
                          + miss-rate * lower-level-access-time

Miss penalty is defined as the difference between lower level access time and cache access time. Then the above equation becomes

    effective-access-time = cache-access-time + miss-rate * miss-penalty

Due to locality of reference, many requests are not passed on to the lower level store. The following equation gives an approximation to the traffic to the lower level.

    lower-level-traffic = miss-rate * requested-traffic

In order to make this more accurate, consideration must be given to data that is fetched from the lower level but not used by the requester. The requester might not need all of the data in a cache line. In any case, the lower level must have the throughput capacity to support the lower level traffic.