Lytle was born on October 20, 1953 to O'Neal and Theresa (Jacobs) Stephen at Harlem Hospital in Harlem, NY. Lytle attended the New York City Public School System. He went on to graduate from Mt. Mansfield High School in Burlington, VT. Lytle was united in marriage to Jo Anne DePalma on January 25, 1983. Later Lytle decided to make a career in the US Navy where he became a Captain's Cook. This led him and his wife JoAnne to settle in Pensacola, FL.

Lytle had a great sense of humor. He enjoyed laughing and making others laugh. He enjoyed the outdoors and was an avid fisherman. He enjoyed cooking for his family and friends. He was an excellent cook. Lytle was affectionately known as "Pops" to his grandchildren. He loved spending time with them.

Lytle is survived by his wife, JoAnne; daughter, Lytisha Walker (Ja Rodney); sons, Terrell Morin and Lytle Jr.; mother, Theresa Stephen; sisters, Lorraine Hayes and Beverly Beard (Roland); twelve grandchildren and a host of nieces and nephews.

Lytle is preceded in death by his father, O'Neal Stephen; brothers, Johnnie and Eddie Stephen and a sister Diana Stephen.

A Visitation will be 3:15pm until the Memorial Service at 4:00pm Saturday, December 10, 2016 at Harper-Morris Memorial Chapel with Pastor Len Ballenger officiating.