background preloader

Information theory

Information theory
Overview[edit] The main concepts of information theory can be grasped by considering the most widespread means of human communication: language. Two important aspects of a concise language are as follows: First, the most common words (e.g., "a", "the", "I") should be shorter than less common words (e.g., "roundabout", "generation", "mediocre"), so that sentences will not be too long. Such a tradeoff in word length is analogous to data compression and is the essential aspect of source coding. Second, if part of a sentence is unheard or misheard due to noise — e.g., a passing car — the listener should still be able to glean the meaning of the underlying message. Such robustness is as essential for an electronic communication system as it is for a language; properly building such robustness into communications is done by channel coding. Note that these concerns have nothing to do with the importance of messages. Historical background[edit] With it came the ideas of This is justified because

http://en.wikipedia.org/wiki/Information_theory

Related:  history of Cognitive PsychologyQuantum Information

Donald Broadbent Donald Eric (D.E.) Broadbent FRS[1] (Birmingham, 6 May 1926 – 10 April 1993) was an influential experimental psychologist from the U.K.[2] His career and research bridged the gap between the pre-WWII approach of Sir Frederic Bartlett[3] and what became known as Cognitive Psychology in the late 1960s. Broadbent's Filter Model of Attention[edit] Broadbent's Filter Model of Attention proposes the existence of a theoretical filter device, located between the incoming sensory register, and the short-term memory storage. His theory is based on the multi-storage paradigm of William James (1890) and the more recent 'multi-store' memory model by Atkinson & Shiffrin in 1968. This filter functions together with a buffer, and enables the subject to handle two kinds of stimuli presented at the same time.

No-communication theorem In physics, the no-communication theorem is a no-go theorem from quantum information theory, which states that, during measurement of an entangled quantum state, it is not possible for one observer, making a measurement of a subsystem of the total state, to communicate information to another observer. The theorem is important because, in quantum mechanics, quantum entanglement is an effect by which certain widely separated events can be correlated in ways that suggest the possibility of instantaneous communication. The no-communication theorem gives conditions under which such transfer of information between two observers is impossible. These results can be applied to understand the so-called paradoxes in quantum mechanics, such as the EPR paradox, or violations of local realism obtained in tests of Bell's theorem. Informal Overview[edit] The theorem is built on the basic presumption that the laws of quantum mechanics hold.

Phronesis Phronēsis ( Greek : φρόνησις) is the Greek word for wisdom or intelligence which is a common topic of discussion in philosophy . In Aristotelian Ethics , for example in the Nicomachean Ethics it is distinguished from other words for wisdom and intellectual virtues – such as episteme and techne – as the virtue of practical thought. For this reason, when it is not simply translated by words meaning wisdom or intelligence, it is often translated as " practical wisdom ", and sometimes (more traditionally) as " prudence ", from Latin prudentia . Phronesis is also sometimes spelled Fronesis .

Chapter 5. Hacker Writing Style We've already seen that hackers often coin jargon by overgeneralizing grammatical rules. This is one aspect of a more general fondness for form-versus-content language jokes that shows up particularly in hackish writing. One correspondent reports that he consistently misspells ‘wrong’ as ‘worng’. Others have been known to criticize glitches in Jargon File drafts by observing (in the mode of Douglas Hofstadter) “This sentence no verb”, or “Too repetetetive”, or “Bad speling”, or “Incorrectspa cing.” Similarly, intentional spoonerisms are often made of phrases relating to confusion or things that are confusing; ‘dain bramage’ for ‘brain damage’ is perhaps the most common (similarly, a hacker would be likely to write “Excuse me, I'm cixelsyd today”, rather than “I'm dyslexic today”). This sort of thing is quite common and is enjoyed by all concerned.

World War II World War II (WWII or WW2), also known as the Second World War, was a global war that lasted from 1939 to 1945, though related conflicts began earlier. It involved the vast majority of the world's nations—including all of the great powers—eventually forming two opposing military alliances: the Allies and the Axis. It was the most widespread war in history, and directly involved more than 100 million people from over 30 countries. In a state of "total war", the major participants threw their entire economic, industrial and scientific capabilities behind the war effort, erasing the distinction between civilian and military resources. World War II altered the political alignment and social structure of the world. The United Nations (UN) was established to foster international co-operation and prevent future conflicts.

Density matrix Explicitly, suppose a quantum system may be found in state with probability p1, or it may be found in state with probability p2, or it may be found in state with probability p3, and so on. The density operator for this system is[1] Human-based computation Human-based computation (HBC) is a computer science technique in which a machine performs its function by outsourcing certain steps to humans. This approach uses differences in abilities and alternative costs between humans and computer agents to achieve symbiotic human-computer interaction. In traditional computation, a human employs a computer[1] to solve a problem; a human provides a formalized problem description and an algorithm to a computer, and receives a solution to interpret. Human-based computation frequently reverses the roles; the computer asks a person or a large group of people to solve a problem, then collects, interprets, and integrates their solutions. Early work[edit] Human-based computation (apart from the historical meaning of "computer") research has its origins in the early work on interactive evolutionary computation.

Digital physics Digital physics is grounded in one or more of the following hypotheses; listed in order of decreasing strength. The universe, or reality: History[edit] The hypothesis that the universe is a digital computer was pioneered by Konrad Zuse in his book Rechnender Raum (translated into English as Calculating Space). The term digital physics was first employed by Edward Fredkin, who later came to prefer the term digital philosophy.[3] Others who have modeled the universe as a giant computer include Stephen Wolfram,[4] Juergen Schmidhuber,[5] and Nobel laureate Gerard 't Hooft.[6] These authors hold that the apparently probabilistic nature of quantum physics is not necessarily incompatible with the notion of computability.

Artificial intelligence AI research is divided into subfields[6] that focus on specific problems, approaches, the use of a particular tool, or towards satisfying particular applications. The central problems (or goals) of AI research include reasoning, knowledge, planning, learning, natural language processing (communication), perception and the ability to move and manipulate objects.[7] General intelligence is among the field's long-term goals.[8] Approaches include statistical methods, computational intelligence, and traditional symbolic AI. Many tools are used in AI, including versions of search and mathematical optimization, logic, methods based on probability and economics. The AI field draws upon computer science, mathematics, psychology, linguistics, philosophy, neuroscience and artificial psychology. History[edit]

Quantum teleportation Quantum teleportation is a process by which quantum information (e.g. the exact state of an atom or photon) can be transmitted (exactly, in principle) from one location to another, with the help of classical communication and previously shared quantum entanglement between the sending and receiving location. Because it depends on classical communication, which can proceed no faster than the speed of light, it cannot be used for superluminal transport or communication of classical bits. It also cannot be used to make copies of a system, as this violates the no-cloning theorem.

Codognet states, "Information theory can be thought of as a sort of simplified or idealized semiotics: a ciphering/deciphering algorithm represents the interpretation process used to decode some signifier (encoded information) into some computable signified (meaningful information) to be fed to a subsequent processing step. This process, like semiosis itself, is, of course unlimted." by arlene Mar 24

Related:  The History of Digital ComputingCommunication EngineeringGovernmentInformationTheories in Engineering