When somone in the investment world refers to "generating alpha" or "beating alpha" they are referring to out-performing a given market indexes annual performance. Alpha is a performance indicator and a component of MPT (Modern Potfolio Theory). The objective of modern portfolio theory (MPT) which was developed in the 1950's is to find the best possible diversification strategies with the objective of achieving a greater expected return with a lower amount of risk. In order to do this it evaluates the following five risk variables:

The overall objective of MPT, as it has evolved, is for investors or rather investment managers/advisors to find a combination of non-correlated investments that have the best potential expected level of return for an appropriate level of risk. 

Investors want to be able to assess the level of performance vs. the risk they are taking. If an investor is taking outsized risk they will expect outsized performance in the event the investment performs as anticipated. If that is not the case, one would naturally have to assess whether taking outsized risk is worth it. 

Alpha is measured in percentage terms. For example, a positive alpha of 2% means that the investment has out-performed a benchmark index by 2%. A negative alpha of 1% means an investment has under-performed an index by 1%.

Alpha is therfore a comparative performance measure that is often used to rank mutual funds and other types of investments.

A more in-depth analysis of 'alpha" may also include "Jensen's Alpha" which takes the capital asset pricing model into account and includes a risk adjusted component in terms of how it is calculated. Apha and Beta are often used together by investment managements to compare and analyze returns.