On his release he joined the Fellowship of Reconciliation (FOR) and the Congress on Racial Equality (CORE). In early 1947, CORE announced plans to send eight white and eight black men into the Deep South to test the Supreme Court ruling that declared segregation in interstate travel unconstitutional. organized by George Houser and Bayard Rustin, the Journey of Reconciliation was to be a two week pilgrimage through Virginia, North Carolina, Tennessee and Kentucky.
Although Walter White of the National Association for the Advancement of Coloured People (NAACP) was against this kind of direct action, he volunteered the service of its southern attorneys during the campaign. Thurgood Marshall, head of the NAACP's legal department, was strongly against the Journey of Reconciliation and warned that a "disobedience movement on the part of Negroes and their white allies, if employed in the South, would result in wholesale slaughter with no good achieved."
The Journey of Reconciliation began on 9th April, 1947. The team included Igal Roodenko, George Houser, Bayard Rustin, James Peck, Joseph Felmet, Nathan Wright, Conrad Lynn, Wallace Nelson, Andrew Johnson, Eugene Stanley, Dennis Banks, William Worthy, Louis Adams, Worth Randle and Homer Jack.
James Peck was arrested with Bayard Rustin and Andrew Johnson in Durham. After being released he was arrested once again in Asheville and charged with breaking local Jim Crow laws. In Chapel Hill Peck and four other members of the team was dragged off the bus and physically assaulted before being taken into custody by the local police.
Members of the Journey of Reconciliation team were arrested several times. In North Carolina, two of the African Americans, Bayard Rustin and Andrew Johnson, were found guilty of violating the state's Jim Crow bus statute and were sentenced to thirty days on a chain gang. However, Judge Henry Whitfield made it clear he found that behaviour of the white men even more objectionable. He told Felmet and Igal Roodenko: "It's about time you Jews from New York learned that you can't come down her bringing your niggers with you to upset the customs of the South. Just to teach you a lesson, I gave your black boys thirty days, and I give you ninety."
The Journey of Reconciliation achieved a great deal of publicity and was the start of a long campaign of direct action by the Congress of Racial Equality. In February 1948 the Council Against Intolerance in America gave George Houser and Bayard Rustin the Thomas Jefferson Award for the Advancement of Democracy for their attempts to bring an end to segregation in interstate travel.
Joseph Felmet died in 1989.