In 1962 Kaukonen moved to the San Francisco Bay Area and enrolled in Santa Clara University. During this time he taught guitar lessons at Benner Music Company in San Jose. As a self-described blues purist, Kaukonen never had any ambition to play in a rock band. He played as a solo act in coffee houses and can be heard accompanying a young Janis Joplin on acoustic guitar on an historic 1964 recording (known as "The Typewriter Tapes" because of the obtrusive sound of Kaukonen's first wife Margareta typing in the background). Invited to attend a Jefferson Airplane rehearsal by founding member Paul Kantner, Kaukonen found his imagination excited by the arsenal of effects available to electric guitar and later said, "I was sucked in by technology."
Kaukonen's electric guitar work was distinctive and widely emulated by other Bay Area guitarists. Notable work with Jefferson Airplane includes "Greasy Heart", "If You Feel", "Hey Frederick" (which culminates in an extended lead guitar duet with himself), "Wooden Ships", and his original composition, "Feel So Good". Rolling Stone named Kaukonen the 54th greatest rock guitarist of all time and 16th greatest acoustic guitarist.
Though never a prolific singer and songwriter during his Airplane tenure, Kaukonen contributed some distinctive material. The album Embryonic Journey showcased his fingerstyle acoustic guitar virtuosity. On the next album, After Bathing at Baxter's, his sound had a harder edge inspired by Mike Bloomfield of the Paul Butterfield Blues Band, Cream, and other groups that visited San Francisco. These stylistic changes are prominent in the acid rocker, "The Last Wall of the Castle", as well as the long instrumental (9:12), "Spare Chaynge", cowritten with bassist Jack Casady and drummer Spencer Dryden. The improvisation marking "Spare Chaynge" is also present in the free-form extended jams, "Thing" and "Bear Melt", both live instrumentals recorded in 1968. Kaukonen insists, however, in the liner notes of the Live at the Fillmore East album, that these jams were not chaotic "free for alls" but in fact "complex rehearsed arrangements." Two songs that were later to become Hot Tuna signature tunes were also recorded during the 1968–1969 period. These were the traditionals, "Rock Me Baby" and "Good Shepherd", a gospel ballad.
Original compositions by him included on the 1971 album, Bark are two instrumentals, "Wild Turkey" and "Feel So Good", and the acoustic autobiographical "Third Week in the Chelsea" detailing his feelings about the disintegration of the band.
In 1969–70, Kaukonen and Jack Casady formed Hot Tuna, a spinoff group that allowed them to play as long as they liked. An early incarnation of Hot Tuna included Jefferson Airplane vocalist Marty Balin and featured Joey Covington on drums and vocals. This grouping came to an end after an unsuccessful recording jaunt to Jamaica, the sessions of which have never been released. Pared down to Kaukonen and Casady, Hot Tuna lived on as a vehicle for Kaukonen to show off his Piedmont style acoustic blues fingerpicking skills. The self titled first album was all acoustic and recorded live. With the dissolution of Jefferson Airplane in 1972, Hot Tuna went electric, with Airplane fiddler Papa John Creach joining for the next two albums. Hot Tuna scored an FM hit with "Ja Da (Keep On Truckin')" from their third (and first studio) album, Burgers. At this time, Kaukonen's songwriting began to dominate, as further evidenced by the next album, The Phosphorescent Rat, which only featured one cover song. Beginning with their fifth album, America's Choice (1974), the addition of drummer Bob Steeler encouraged a rise in volume and a change of band personality —a rampaging, Cream-like rock with often quasi-mystical lyrics by Kaukonen. During this period, the power trio was known for its very long live sets and instrumental jamming.
In 1974, Kaukonen recorded the first and most successful of several solo albums, Quah, together with Tom Hobson. Produced by Jack Casady, and featuring (somewhat surprisingly) string overdubs on some tracks, this album contained some of Kaukonen's most deft fingerpicking work, especially on "Hamar Promenade", "Blue Prelude", "Genesis" and " Flying Clouds". The curious picture that adorns Quah's cover is today on display at Donkey Coffee and Espresso, a coffee shop in Athens, Ohio.
Kaukonen toured vigorously throughout the 1970s in both the United States and Europe, but with Hot Tuna's break up in 1978, the first phase of the band's career ended. Casady left to form the new wave band "SVT", while Kaukonen released his second solo album, Jorma, a mix of electric guitar and acoustic fingerstyle in 1979. Meanwhile, he had formed the band "Vital Parts."
Vital Parts featured bassist Denny DeGorio, who had played in a San Francisco band called the "Offs" with ex-Hot Tuna drummer Bob Steeler. Kaukonen, experimenting with a new image, not only cut his hair but dyed it purple then bright orange, and had extensive tattoos adorn his body, back and arms. The album Barbeque King was released in 1980. Kaukonen's traditional fan base did not warm to this new, perceived to be "punk" image, and sales of the album were so disappointing that Jorma was soon dropped from RCA Records.
He continued playing as a solo artist throughout the 1980s at such venues as The Chestnut Cabaret in Philadelphia, The Capitol Theater in Passaic, New Jersey and in Port Chester, New York. As in his Hot Tuna days, he played very long sets, usually beginning with an hour-long acoustic set followed by a long intermission and then a two hour electric set, sometimes accompanied by bass and drums. Hot Tuna themselves reformed in the late 1980s. At a 1988 Hot Tuna performance at the Fillmore Auditorium, Kaukonen surprised fellow Airplane alumnus Paul Kantner, who was sitting in, with a surprise appearance by his estranged lover Grace Slick; the success of this performance helped to pave the way for a Jefferson Airplane reunion tour and record in 1989.
In 1984, Kaukonen appeared on Robert Hunter's Amagamalin Street. This was the third album released by Relix Records, a label, founded by Les Kippel, that specialized in bands from the San Francisco Bay Area. Relix also released Splashdown, featuring a rare performance by Hot Tuna on WQIV, a now-defunct radio station in New York. Kippel was instrumental in reuniting Kaukonen and Casady in 1985 for a Hot Tuna theater tour. Relix Records remained Hot Tuna's record label until 2000, and also released Classic Hot Tuna Acoustic, Classic Hot Tuna Electric, Live at Sweetwater, and Live at Sweetwater Two.
Two notable projects featuring Kaukonen were David Crosby's debut solo album If I Could Only Remember My Name, on Atlantic (1971) and Warren Zevon's Transverse City on Virgin in 1989. In 1993, he collaborated with ex-Grateful Dead keyboardist Tom Constanten in recording numerous arrangements of "Embryonic Journey". The resulting tracks were released as Embryonic Journey, the album, in 1994 on the Relix label. In 1999, he played several gigs with Phil Lesh and Friends. In 2000, he appeared with jam band Widespread Panic during their summer tour.
With his wife Vanessa, Kaukonen currently owns and operates the Fur Peace Ranch, a 119 acre music and guitar camp in the hills of southeast Ohio, north of Pomeroy; complete with a 32 track studio. He is currently under contract as a solo artist to Red House Records and still records and tours with Jack Casady and other friends such as Barry Mitterhof as Hot Tuna. His 2002 album Blue Country Heart, also released as a 5.1 single layer SACD, was widely acclaimed by critics as one of the definitive examples of American "Depression Era " music and features Kaukonen backed by an all-star Nashville bluegrass band. The album was nominated for a Grammy Award.
Recent solo albums include Stars in My Crown (2007) and River of Time (2009).