Yevamos, Chapter 2, Mishnah 5. The Mishnah opens by establishing: "Mi she'yesh lo ach mikol makom - zokek es eishes achiv liyibum" - one who has a brother of any kind binds his brother's wife to yibum. In other words, the existence of a brother creates the obligation of yibum upon his brother's widow.
"Ach mikol makom" - Including a Mamzer:
The Gemara asks: what is meant by the phrase "a brother of any kind"? It comes to include even a mamzer. A mamzer is one born of a forbidden relationship, and his status means that he cannot marry a Jewish woman, only a female mamzeres like himself or a convert - and even that union produces further mamzerim. Nevertheless, he too is considered a brother for all the laws of yibum: if the only surviving brother is a mamzer, he binds the widow to yibum. True, she cannot actually marry him, and therefore she will require chalitzah alone, but the bond itself exists.
"Vaharei hu achiv l'chol davar":
The Mishnah continues that this brother is considered his brother in every respect, and the Gemara explains that this refers to two matters:
"L'yorsho" - they inherit from one another, when there is no other person who takes precedence in inheritance.
"Ul'tam'o" - if one of them dies, his brother is obligated to become tamei for the deceased, even if he is a kohen. A kohen who has a brother who is a mamzer is obligated to take part in his burial and his funeral.
"Chutz mimi she'yesh lo min hashifchah umin hanochris":
The exception is a brother born from the same father, but whose mother was a maidservant or a non-Jewish woman - such people are not considered brothers at all, and there is no relationship between them of the kind that exists between brothers. The reason is that such a son has no lineage traced to his father, and the father is the only link through which they would be connected. Since the son is not attributed to the father, he is likewise not considered a brother to the father's other sons.
"Mi she'yesh lo ben mikol makom":
The Mishnah continues: "Mi she'yesh lo ben mikol makom - poter eishes aviv min hayibum" - one who has a son of any kind exempts his father's wife from yibum. A man who has a son of any sort, even a mamzer, creates a situation in which his father's wife is exempt from yibum. For the obligation of yibum applies only when a man dies without children; once he has left behind a son, even a mamzer, there is no obligation of yibum.
"V'chayav al makaso v'al kilelaso":
Moreover, such a son is liable to punishment for striking his father and for cursing him - two transgressions that carry the death penalty. Even though the relationship between them was founded on an act of disqualification, a father-son relationship still exists between them in every respect.
"Uv'no l'chol davar":
And he is his son for every purpose - including inheritance, that he inherits from him; and likewise regarding tumah, that if the father is a kohen, he is obligated to become tamei for him.
"Chutz mimi she'yesh lo ben min hashifchah umin hanochris":
Once again, the exception is a son born from a maidservant or from a non-Jewish woman - since in such a case there is no bond of lineage between the father and the son.
In summary: In this Mishnah we learned that the phrase "of any kind" comes to include even a mamzer - both regarding the brother, who binds his brother's widow to yibum (and in practice to chalitzah), and regarding the son, who exempts his father's wife from yibum. Such a brother and such a son are considered relatives in every respect: for inheritance, for becoming tamei for the deceased, and for liability for striking and cursing one's father. The exception is a son born from a maidservant or from a non-Jewish woman, who has no lineage traced to his father, and consequently there is no father-son relationship or brotherhood between them.