Pickens, who epitomized the profane, tough, sardonic cowboy, was born in Kingsburg, California. He was an excellent rider from age four and quit school to join the rodeo at age twelve. He was told that working in the rodeo would be "slim pickings", giving him his name, but he did very well, eventually rising to become a well known rodeo clown - one of the most dangerous jobs in show business.

After twenty years on the rodeo circuit, his distinctive voice and drawl, his wide eyes and moon face, and his strong physical presence and grace gained him a role in the western Rocky Mountain (1950), starring Errol Flynn. He subsequently appeared in many westerns, playing both villains and comic sidekicks to the likes of Rex Allen. In the opening scene of An Eye for an Eye (1966), he shoots a baby in its crib.

His most famous role was as B-52 pilot Major T. J. "King" Kong in Dr. Strangelove, which ended with Pickens riding an H-bomb down to global destruction.

In one scene, Pickens briefs the crew on their survival packs:

"In them you'll find one .45-caliber automatic, two boxes ammunition, four days' concentrated emergency rations, one drug issue containing antibiotics, morphine, vitamin pills, pep pills, sleeping pills, tranquilizer pills, one miniature combination Russian phrase book and Bible, one hundred dollars in rubles, one hundred dollars in gold, nine packs of chewing gum, one issue prophylactics, three lipsticks, three pair of nylon stockings ... Shoot, a fella could have a pretty good weekend in Vegas with all that stuff." "Dallas" in the script; later redubbed after Kennedy was shot there.

Another of his memorable roles was as Taggart, head of the gang of cowboy thugs in Mel Brooks' classic 1974 comedy Blazing Saddles:

"What in the Wide Wide World of Sports is a-goin' on here, I hired you people to try to git a little track laid, not to jump around like a bunch of Kansas City faggots!" The next year, Pickens was in another western, playing the evil limping bank robber in Walt Disney's The Apple Dumpling Gang

Pickens lent his voice to the 1938 children's radio show The Cinnamon bear, where he plays a singing cowboy.

Pickens appeared in dozens of films, including, Old Oklahoma Plains (1952), Down Laredo Way (1953), Major Dundee (1959) with Charlton Heston, One-Eyed Jacks (1961) with Marlon Brando, The Cowboys (1972) with John Wayne, Ginger in the Morning (1974) with Fred Ward and The Getaway1979) with Steve McQueen.

Pickens was offered the part of Dick Hallorann in Stanley Kubrick's 1980 adaptation of Stephen King's The Shining. He refused, saying that filming with Kubrick on Dr. Strangelove was too strenuous. He later relented, saying that he would appear in the film as long as Kubrick was contractually required to shoot Pickens's scenes in fewer than 100 takes a shot. Kubrick, notorious for shooting scenes hundreds and hundreds of times, refused, and cast Scatman Crothers as Hallorann instead.

He also appeared many times on television, both in guest shots, and in regular roles in The Legend of Custer, Bonanza, B.J. and the Bear, and Filthy Rich (1982). He played the owner of station WJM, Wild Jack Monroe, on the Mary Tyler Moore Show.

In 1982, he was inducted into the Western Performers Hall of Fame at the National Cowboy & Western Heritage Museum in Oklahoma City, Oklahoma. A year later, he died from a brain tumor at the age of 64. Pickens was living at the Evergreen nursing home in Modesto, California after his brain surgery and that was where he passed away.

His brother acted under the name Easy Pickens. His most notable appearance was as "Easy" in Sam Peckinpah's The Ballad of Cable Hogue (1970).