Environment for creative processing of text and numerical data

Symmetric random walk

By detecting that (by rotation of 45 degrees) the symmetric random walk in the plane can be seen as a combination of two independent and simultaneous one-dimensional random walks, it was easier to study asymptotic properties.

Reversing

Birds (word and phrase continuation)

There has been an context sensitive autotext feature in the Survo Editor already from 1989 by means of the key combination F2 J.

F1 J is an extended alternative for F2 J for completing phrases found elsewhere in the current edit field. As in this example, a list containing 'all possible phrases' may be loaded to the end of the edit field. This list then serves as a source of information during writing process by giving synonyms, technical terms etc.

It is easy to create such lists on different topics by pasting them from websites, for example. The list used in this example originates from Birds of Sweden.

Graphical rotation in factor analysis

This feature has been available already in SURVO 76.

The traditional graphical rotation is described e.g. in
Ledyard Tucker and Robert MacCallum: Exploratory Factor Analysis, Chapter 10.

The principles of Cosine rotation and Transformation analysis were introduced in Yrjö Ahmavaara's dissertation
Transformation Analysis of Factorial Data, Helsinki Ann. Acad. Sci. Fenn., B 88, 2, 1954.

The current algorithm for Cosine rotation was created in 1961 and described in Matrix computations in Survo.

Pythagorean points on a green meadow

A comprehensive documentation is given in my paper Visualization and characterization of Pythagorean triples.
Interactive 'graphical' identifying of Pythagorean triples is available as a sucro /P_TRIPLE.

Unbiased coin-flips with a biased coin

It is surprising that a biased coin (even without knowing its probabilities p for HEADS p and 1-p for TAILS) may be used like a fair coin by observing coin-flips in pairs. The expected number of flips of the biased coin for extracting one unbiased coin-flip is 1/(p*(1-p)) which gets its minimum value 4 for p=1/2, i.e. when the coin actually is fair. Thus typically four or more flips are needed if we do not believe that the coin is fair.

The original data of 1200 flips with a biased coin (p=1/3) was generated by Survo as follows:

FILE CREATE COIN,1,1
FIELDS:
1 N 1 X
END

FILE INIT COIN,1200

p=1/3
MATRIX P ///
0 p
1 1-p

MAT SAVE P

RND=URAND(20106)
TRANSFORM COIN BY #DISTR(P)
MAT SAVE DATA COIN AS COIN2
MAT COIN3=VEC(COIN2,20)      / *COIN3~VEC(COIN2) 20*60

MAT LOAD COIN3,##,CUR+1
MATRIX COIN3
VEC(COIN2)
///       1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 ...
  1       1  0  0  1  1  1  1  1  0  1  1  1  0  0  1  1  1 ...
  2       0  1  1  1  0  1  0  1  1  1  1  1  0  1  1  1  0 ...
  3       1  1  1  1  0  1  1  1  1  0  1  1  1  1  1  1  1 ...
  4       1  0  1  1  1  1  1  0  1  1  1  0  1  0  1  1  1 ...
  5       1  1  1  1  1  1  1  1  0  1  1  1  1  0  1  1  1 ...
  .       .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  . ...

Omega coin tossing

This approach is based on the following facts:
An integer can be decomposed into prime factors in only one way.
There is strong number-theoretic evidence for the fact that for (large) integers the number of prime factors is even or odd with equal probabilities. This is also intuitively obvious.

To give credence to the fact that the number of prime factors is even or odd with probablity 1/2, I took a random sample of 10 million integers with 16 digits by the following Mathematica code

a=10^15
SeedRandom[1];
t1=TimeUsed[];
tab:=Table[PrimeOmega[RandomInteger[{a,a+8999999999999999}]],{n,1,10^7}];
Export["Sample.txt",tab,"Table"]
TimeUsed[]-t1

Omega      f     %      *=65536 obs.
  1   277575   2.8 ****
  2  1053208  10.5 ****************
  3  1860960  18.6 ****************************
  4  2105565  21.1 ********************************
  5  1783889  17.8 ***************************
  6  1245149  12.5 ******************
  7   765283   7.7 ***********
  8   433642   4.3 ******
  9   232340   2.3 ***
 10   120919   1.2 *
 11    60947   0.6 :
 12    30650   0.3 :
 13    15225   0.2 :
 14     7482   0.1 :
 15     3655   0.0 :
 16     1814   0.0 :
 17      876   0.0 :
 18      408   0.0 :
 19      217   0.0 :
 20      102   0.0 :
 21       49   0.0 :
 22       22   0.0 :
 23       15   0.0 :
 24        3   0.0 :
 25        2   0.0 :
 27        1   0.0 :
 28        1   0.0 :
 29        1   0.0 :

In number theory the function lambda(n)=(-1)^Omega(n) getting "randomly" values -1 and +1 is known as Liouville function and it completely corresponds to "Omega coin tossing". The sums of lambda(n) has been studied experimentally in Sign changes in sums of the Liouville function by Borwein, Ferguson, and Mossinghoff (2010).

Rational approximations by listening

Musicians (who often claim that they know nothing about mathematics) are clever in recognizing intervals and chords even when the sound is not pure. Thus when hearing an interval x as a major third they unconsciously find that 5/4 is the best approximation for x.

The VAR and PLAY commands of Survo give an opportunity to create and play sound files (in WAV format). For example, the neutral third and pure intervals close to can be listened as follows:

s(X):=sin(X*ORDER)

f1=11/9 1.2222222222222...          Interval_11_9
f2=sqrt(5/4*6/5) 1.2247448713916... Neutral_third
f3=5/4 1.25                         Major_third
f=0.2 'basic frequency'

FILE MAKE Test,1,24000,X,2            / creates data file Test.
VAR X=10000*(s(f)+s(f*f1)) TO Test    / computes the wave form
PLAY DATA Test,X / WAV=Interval_11_9  / converts the wave form into WAV
VAR X=10000*(s(f)+s(f*f2)) TO Test
PLAY DATA Test,X / WAV=Neutral_third
VAR X=10000*(s(f)+s(f3*f)) TO Test
PLAY DATA Test,X / WAV=Major_third

PLAY SOUNDS                           / plays sound files created
Interval_11_9
Neutral_third
Major_third

Slightly off the topic:
The neutral third sqrt(3/2) is recognized correctly from its
approximate value 1.2247448713916 by the INTREL command
INTREL 1.2247448713916 / giving
X=1.2247448713916 is a root of 2*X^2-3=0

Color changing

The graph is a modification of an old Survo application presented in User Guide (1992) on page 334. This 'animated' version is inspired by Life A User's Manual (La Vie mode d'emploi) by Georges Perec.

In the plotting scheme the jumps are triggered by int() functions in X(t) and Y(t) expressions.

HEADER=  HOME=0,0 WHOME=0,0 WSIZE=652,381 WSTYLE=0 MODE=652,381
XSCALE=-14,14 YSCALE=-10,10 FRAME=3
SIZE=652,381 XDIV=0,1,0 YDIV=0,1,0
a=53 t=0,50*pi,pi/150 pi=3.14159
GPLOT X(t)=12*sin(int(0.35*t))+0.8*sin(a*t)*sin(t)+0*A,
      Y(t)=8*sin(int(0.3*t))+0.8*sin(a*t)*cos(t)
A=0(1)4
PALETTE=BGY3 COLORS=[background(8)] COLOR_CHANGE=A,3
SLOW=100 slowing down the plotting speed

Permutation test

This is an abbreviated version of a Finnish teaching program made in 1998. I wanted to show how simple is the 'theory' behind randomization tests when compared to that of the t test, for example, and how a time-consuming permutation test can be replaced by its randomized alternative.

The COMB operation and the Survo matrix interpreter are the key components in this experiment. Also a few other special features of Survo are applied.

HH - HT game (analysis)

This is an abbreviated version of a Finnish teaching program made in 1998. It is shown how the winning probabilities and expected values can be derived simply by considering them conditionally with respect to the first pair of coin-flips. The probabilities are found by solving a system of four linear equations.

This example is continued by a simulation experiment HH - HT game (simulation).

HH - HT game (simulation)

This is an abbreviated version of a Finnish teaching program made in 1998 and continuation of HH - HT game (analysis).

Solving a Survo puzzle

I presented Survo Puzzle in 2006. More information is available on the home page.

The puzzle solved here has a moderate degree of difficulty (400). It was selected randomly by using a Java applet. By clicking the applet and entering the serial number #352-23824, this Survo puzzle can be solved more easily by the Swapping method. (This is shown in another demo.) But when using this technique the uniqueness of the solution cannot be confirmed.

Survo supports solving of these puzzles in many ways. Besides the COMB operation also Editorial computing and Touch mode are useful tools. The editorial interface of Survo is also suitable for general book-keeping during the solving process.

Lines going through 3 points in a 9x9 grid

This is an illustration related to my paper On lines through a given number of points in a rectangular grid of points. On the cover page of that paper a picture of all L(16,4)=548 lines connecting exactly four points in a 16x16 grid is presented.

Although efficient computing of numbers L(n,j), i.e. # of lines going through j points in and nxn grid of points, is not trivial, making a list of these lines is still more demanding task for large n values. However, such a list for small n is easy to generate by brute force on current computers. Thus the Survo program module GRIDP simply starts from all pairs of points in the grid and sorts out the required lines.

The graph is a typical example of how in Survo complicated pictures are compiled of several overlaying parts, here starting from a black background, then drawing the line segments, and finally setting the points as small 'hollow' circles.

A related example

Solving a Survo puzzle by the swapping method

When a Survo puzzle is solved by the swapping method there is no guarantee that the solution is the only possible. The same puzzle is solved in another demo systematically showing at the same time that the solution is unique.

If this puzzle is given as an open Survo puzzle without any fixed numbers in the form

    A  B  C  D  E
 1  *  *  *  *  * 41
 2  *  *  *  *  * 28
 3  *  *  *  *  * 51
   34  8 13 28 37

Prime numbers listed by a sucro

The prime numbers are found by using a variation of the 'trial division' method:
Given a number n, one divides n by all numbers m less than or equal to the square root of that number. If any of the divisions come out as an integer, then the original number is not a prime. Otherwise, it is a prime.

After numbers 2, 3, and 5 are listed, both n values (Wnumber in PRIMES) and m values (Wfactor) are selected so that they are not divisible by 2 or 3.

A sucro program cannot be efficient in purely numerical problems since all objects processed by a sucro are presented as strings of characters. The 'values' of variables are saved in a 'sucro memory' which is simply a string. For example, at the end of the current application this string is

5000@5003@71@29@4@2@5041@
giving values
WN=5000
Wnumber=5003 (first integer exceeding 5000: 4999+Wi)
Wdivisor=71
Wremainder=29 (last accepted prime 4999 mod 71 is 29)
Wi=4 (oscillating between 2 and 4)
Wj=2 (similarly)
Wsquare=5041 (71^2=5041)

It is evident that repeating string conversions to numerical values and vice versa are slowing down the speed of computation.

In typical applications of sucros, like teaching programs and demos (like these GIF animations) and combining several Survo operations, this feature is unimportant. Although Survo program modules are written in C, many system routines are sucros.

A general description of the sucro language is given in User Guide (1992) chapter 12 (pages 399 - 443).

Ulam spiral in color

Background information about the Ulam spiral in Wikipedia, for example.

When making the spiral the main task is to map values of n to x,y coordinates. I derived the formulas
x(n)=x(n-1)+sin(mod(int(sqrt(4*(n-2)+1)),4)*pi/2)
y(n)=y(n-1)-cos(mod(int(sqrt(4*(n-2)+1)),4)*pi/2)
by observing that the turning points of the spiral may be described in this way

           12 11 11 11 11 11 11
           12  8  7  7  7  7 10  .
           12  8  4  3  3  6 10  .
           12  8  4  1  2  6 10 14
           12  8  5  5  5  6 10 14
           12  9  9  9  9  9 10 14
           13 13 13 13 13 13 13 14

1,2,3,3,4,4,5,5,5,6,6,6,7,7,7,7,8,8,8,8,9,9,9,9,9,... with a general term a(n)=int(sqrt(4n+1)), n=0,1,2,...
This is verified easily by observing that a(n) grows exactly on values n=k^2 and n=k(k+1), k=0,1,2,... and then the gaps between growing points are 1,1,2,2,3,3,4,4,... leading to the sequence in question.
The increments in x coordinates are 1,0,-1,-1,0,0,1,1,1,0,0,0, following the same pattern of value changes as a(n) but with cyclic variation 1,0,-1,0 of length 4. Therefore the increment in x values can be expressed as sin(mod(int(sqrt(4*(n-2)+1)),4)*pi/2). This is a bit overuse of trigonometric functions but nice for the VAR operation of Survo. The increments of the y coordinates follow the same pattern by changing sin to -cos.

Testing the correlation coefficient

This is a typical example of how by means of editorial computing and text processing a general computation scheme is created for a particular application.

The Fisher zeta transformation makes the correlation coefficient approximately normally distributed with standard deviation 1/sqrt(n-3). The cumulative normal distribution function is available as a library function N.F(m,s^2,x).

If you like to use this template in your own Survo, please copy/paste it from User Guide, page 56.

Age pyramid (Finland 2009)

The age pyramid (TYPE=PYRAMID) is one of types of bar charts in Survo.

Letter frequencies in Shakespeare's Sonnets

Shakespeare's 154 Sonnets were imported from
http://www.shakespeares-sonnets.com/allsonn.htm.

The same textual data is studied in demos Shakespeare's Sonnets as a Markov chain and The most common words in Shakespeare's Sonnets.

It is shown how Survo can deal with literal and partially disorganized data. In many cases such data sets have to be scanned, filtered, and purified from unsystematic features. In this example, various forms of the LINEDEL command of Survo were useful.

The letter frequencies in Shakespeare's Sonnets counted in this demo seem to deviate significantly from current English at least for some common letters like a,c,h,t as one can see in the following table.

Letter  Sonnets English Difference
          %       %
a       6.8     8.2    -1.4
b       1.7     1.5     0.2
c       1.8     2.8    -1.0
d       3.8     4.3    -0.5
e      12.5    12.7    -0.2
f       2.3     2.2     0.1
g       1.9     2.0    -0.1
h       7.0     6.1     0.9
i       6.4     7.0    -0.6
j       0.1     0.2    -0.1
k       0.8     0.8     0.0
l       4.2     4.0     0.2
m       2.9     2.4     0.5
n       6.2     6.7    -0.5
o       7.8     7.5     0.3
p       1.4     1.9    -0.5
q       0.1     0.1     0.0
r       5.7     6.0    -0.3
s       6.8     6.3     0.5
t       9.9     9.1     0.8
u       3.2     2.8     0.4
v       1.3     1.0     0.3
w       2.6     2.4     0.2
x       0.1     0.2    -0.1
y       2.7     2.0     0.7
z       0.0     0.1    -0.1

Shakespeare's Sonnets as a Markov chain

Shakespeare's 154 Sonnets were imported from http://www.shakespeares-sonnets.com/allsonn.htm.
The same textual data is studied in demos Letter frequencies in Shakespeare's Sonnets and The most common words in Shakespeare's Sonnets.

The simulations were made according to a technique presented by Claude Shannon in A Mathematical Theory of Communication (1948).

Linear regression analysis by orthogonalization

The matrix formulas used in this demo are derived, for example, in Survo User Guide pp. 377-378. The REGDIAG program of Survo uses the same algorithm based on orthogonalization of the regressor matrix.

The automatic labelling of matrix rows and columns has been possible already in the matrix interpreter of SURVO 76 (in 1977). It is important to notice the rules for labels in derived matrices. For example, labels are transposed not only when transposing a matrix but also when matrix is inverted, etc. A simple label 'algebra' ensures that in the matrix of regression coefficients the names of regressors appear as row labels and the names of regressands as column labels.

Most common words in Shakespeare's Sonnets

Shakespeare's 154 Sonnets were imported from http://www.shakespeares-sonnets.com/allsonn.htm.
The same textual data is studied in demos Letter frequencies in Shakespeare's Sonnets and Shakespeare's Sonnets as a Markov chain.

The most important tools were the WORDS, STAT, and SORT commands.

The order of the most common words differs from that of common English for obvious reasons.

Discriminant analysis of Iris flower data set

Multiple discriminant analysis is performed in Survo by computing covariance structures (correlations, means and standard deviations) for each group of observations by the CORR operation. Then the actual analysis takes place using these results by the sucro command /DISCRI. The computations are made automatically by the MAT commands (matrix interpreter) of Survo. /DISCRI saves the results as matrix files and lists suitable commands in the edit field for retrieving them.

One of those commands is
MAT LOAD DISCRXR.M,END+2 / Correlations variables/discriminators
giving in this case

MATRIX DISCRXR.M
Correlations_between_variables_and_discriminators
///        Discr1   Discr2
Sepal_L  -0.79189 -0.21759
Sepal_W   0.53076 -0.75799
Petal_L  -0.98495 -0.04604
Petal_W  -0.97281 -0.22290

The discriminant scores were computed by a LINCO command
LINCO Iris,DISCRL.M(D1,D2)
(as suggested by /DISCRI).

The same data is studied in Cluster analysis of Iris flower data set. by classifying the observations into three groups without any prior information about the species of flowers. It turns out that then clustering according to Wilks' Lambda criterion will be identical to that obtained by reclassification of the original observations according to Mahalanobis distances after discriminant analysis.

Cluster analysis of Iris flower data set

Survo offers alternative means for making cluster analysis. In this case the best result was achieved by statistical clustering based on Wilks' Lambda criterion.

Pekka Korhonen has presented an effective stepwise procedure for computation of lambda values in his doctoral thesis "A stepwise procedure for multivariate clustering", Computing Centre, University of Helsinki, Research Reports N:o 7 (1979).
In Korhonen's research a pivot operation plays an essential part in a form presented earlier by Hannu Väliaho in his doctoral thesis "A synthetic approach to stepwise regression analysis", Comm.Phys.Math., vol.34, No.12, 91-132 (1969).

In the CLUSTER program of Survo, the dual procedure of Korhonen's stepwise method is applied. I was Korhonen's opponent in his dissertation and then I took a task to check his algorithms by implementing them to Survo.

The same data is studied in Discriminant analysis of Iris flower data set

F1 connections

In gif-animated demos (and in particular in this one) the size of the 'window' limits possibilities of showing interplay between Survo and other programs. Only a hands-on approach can tell the whole truth. Recommended!

"Rotated arrowheads"

A new feature (valid in SURVO MM from ver. 3.21) for denoting points by arrowheads in scatter diagrams is presented. New point types are 21 (arrow) and 22 (filled arrow) are available and the orientation of arrows is selected by a variable, say A, giving the direction angle in degrees and determined by a code [rotation(A)] in the POINT specification.

Influence curves for the correlation coefficient

The plotting setup

PLOT z(x,y)=abs(r*(1-w)+u*v)/w
  u=sqrt(n/(n*n-1))*(x-mx)/sx
  v=sqrt(n/(n*n-1))*(y-my)/sy
  w=sqrt((1+u*u)*(1+v*v))
TYPE=CONTOUR SCREEN=NEG ZSCALING=20,0

I presented this example among others in my talk about Survo in Compstat 1992 (Neuchatel).
This graph was used by the organizers of Compstat as a cover page (upside down!!) in the proceedings of the symposium. http://en.wikipedia.org/wiki/Computational_statistics

Colored texts in bar/pie charts

In bar and pie charts, labels of variables can be written in the graph by a LABELS specification and values of variables by a VALUES specification. From the version 3.22 of SURVO MM these texts can be colored individually by using an extended form of the SHADING specification (here on line 8) referring to both fill colors and to text colors (separated by a slash /).

The colors to be used are defined by COLOR(n) specifications telling the color components of each SHADING value n according to the CMYK color model.

"Hello World!"

Virtual keyboard

 
Before SURVO MM (until year 2000) the mouse had practically no role while using Survo. Thereafter natural functions of the mouse were adopted. This example shows how almost everything may now be done without the keyboard just by the mouse and a virtual keyboard. Needless to say, practice of mouse-oriented use to such extent is not very convenient.

The user may edit soft buttons and also create new ones while using Survo. The default set of soft buttons is defined in the edit field <Survo>\U\SUR-SOFT.EDT specifying, for example, the main button line EXIT in the form:
 

Most of the soft buttons lead to activation of a Survo macro (sucro). For example, clicking the START button activates sucro /SURVO-START (see lines 60-61).

Cycloid

A cycloid is a curve defined by the path of a point on the edge of circular wheel as the wheel rolls along a straight line.

The first Survo Editor (1979) was originally programmed for input and editing of musical manuscripts and for converting them into a printable form. The slurs (arched curves connectiong a group of notes) were then plotted as slightly modified cycloids.
See: The origin of Survo Editor

Prime factors of numbers m^n-1

This 67th demo is a tribute to Mr. Cole who found by numerical computations in 1903 that the Mersenne number M67=2^67-1 was not a prime number but a product of integers 193707721 and 761838257287. Finding of these factors was made easier e.g. by the fact that it was known beforehand that each potential factor has the form c*67+1 since 67 is a prime. In this case 193707721=2891160*67+1 and 761838257287=11370720258*67+1.

It seems that, in general, numbers of type m^n-1 typically have many (and sometimes all) prime factors of the form c*n+1. By plain numerical calculations I have tried to study their abudance and found some systematic results reported in my paper. These results may have been proved already before. Thus if somebody knows about such proofs, please, let me know.

Editorial computing in Survo makes inventing and testing of this kind of numerical hypotheses easy and comfortable according to the style used in this demo. Making of suitable sucros (Survo macros) is also helpful. The most important sucro /MPN used in this connection has the following listing in a Survo edit field:

*TUTSAVE MPN
/ /MPN m_max,n                              / SM 4.12.2010
/ assuming that n is a prime number
/ computes the prime factors of numbers (m^n-1)/(m-1) for
/ m=2,3,...,m_max
/ and represents them in the form c*n+1.
/ If m-1 divides n, the smallest factor is n.
/
/ See: ../papers/MustonenPrimes.pdf
/
/ def Wmax=W1 Wn=W2 Wm=W3 Wprod=W4 Wc=W5 Wfactor=W6 Wpow=W7
/
*{init}{tempo 0}{disp off}{Wm=2}{R}
*int(exp(log(9000000000000000)/{print Wn}))={act}{l} {save word Wc}
*{line start}{erase}{u}{disp on}{tempo 2}
- if Wmax <= Wc then goto A
*{Wmax=Wc}
+ A: {R}
*({print Wm}^{print Wn}-1)/({print Wm}-1)={act}{l} {line end}
*(10:factors)={act}
/ Remove text "(10:factors)=":
*{l13}{del12}{r}{ref set 1}
/
*{save line Wprod}{erase}{R}{print Wprod}{line start}
/ Replace *'s by spaces:
+ B: {r}{save char Wc}
- if Wc '=' {sp} then goto C
- if Wc '<>' * then goto B
/ Replace * by a space:
* {goto B}
+ C:
/ Each factor to a separate line:
*{home}{u}{ins line}TRIM 1{act}{del line}{home}
*{save char Wc}
- if Wc '<>' {sp} then goto D
*{del line}
/
+ D: {save word Wfactor}{Wpow=1}
- if Wfactor = 0 then goto G
- if Wfactor > Wn then goto D1
/ Wfactor is n
*{form}{goto F}
+ D1:
/ Search for ^
+ D2: {r}{save char Wc}
- if Wc '=' ^ then goto D3
- if Wc '<>' {sp} then goto D2 else goto D4
+ D3:  {save word Wpow}{home}{save word Wfactor}
+ D4: {line start}{erase}({print Wfactor}-1)/{print Wn}={act}
/
*{l} {save word Wc}{line start}{erase}({print Wc}*{print Wn}+1)
/
*{line start}{save word Wfactor}
+ F: {ref jump 1}{write Wfactor}
- if Wpow = 1 then goto F2
*^{print Wpow}
+ F2: *{ref set 1}{R}{del line}{goto D}
+ G: {ref jump 1}{l}{del}
/
- if Wm = Wmax then goto E
*{Wm=Wm+1}{goto A}
+ E: {end}

Some properties of Magic Squares

This demo is created by Kimmo Vehkalahti.

Functions of the Survo matrix interpreter are shown in connection with Magic Squares.

Some further properties of Magic Squares

This demo is created by Kimmo Vehkalahti.

Functions of the Survo matrix interpreter are shown in connection with Magic Squares.

Solving linear equations

A system of linear equations

X1+X2=27
X2+X3=32
X3+X4=32
X1+X3=25

MATRIX A
///  X1 X2 X3 X4
r12   1  1  0  0
r23   0  1  1  0
r34   0  0  1  1
r13   1  0  1  0

MATRIX B
/// freq
r12   27
r23   32
r34   32
r13   25

Polynomial regression

An essential tool for making polynomial regression is the POWERS program. It computes powers of selected variables up to a given degree as new variables. Thereafter polynomial regression analysis is carried out by standard tools like LINREG.

In this example, polynomial regression is applied for determining unknown coefficients of a certain polynomial of two variables from a sample of values.

Originally, this calculation was presented in my note (in Finnish) (2004) related to computation of a distribution of the city block distance D between two random points in a grid of N x N points.
F(N,K)/N^4 is then the probability P[D=K] for K=1,2,...,N-1.

Marking columns by SHADOW SET

Shadow characters play an essential role in the editorial interface of Survo. In fact, each line in the Survo Editor may have an optional line consisting of shadow characters. Their existence is indicated by various display effects. For example, '1' as a shadow character makes the corresponding (main) character red and when lines are printed, these red characters typically appear in boldface.

Survo has certain tools for management of shadow lines. The SHADOW SET command is the most recent one (included at the end of year 2011). It enables filling columns of tables with selected shadow characters thus enhancing their appearance.

In fact, this new SHADOW SET does exactly the same job for the shadow lines as the 'classic' SET command for ordinary edit lines.

Hunting quanta

Consider a data set x_1, x_2,..., x_n where each observation is an approximate integral multiple of one of positive numbers q_1, q_2,..., q_k where typically k=1 or another small integer.

Our task is to estimate the values of quanta q_1, q_2,..., q_k on the condition that each of them exceeds a certain minimum value q_min.

D.G.Kendall has in his paper Hunting Quanta (Royal Society of London. Mathematical and Physical Sciences A 276, 231-266) proposed using a "cosine quantogram" of the form

                         n
    phi(q) = sqrt(2/n)* SUM cos(2*pi*eps(i)/q)                (Kendall 1974)
                        i=1

My idea is that the quanta are estimated by a selective, conditional least squares method where the sum

                      n
   ss(q_1,...,q_k) = SUM min[g(x_i,q_1)^2,...,g(x_i,q_k)^2]       (SLS 2005)
                     i=1

A more detailed description is found in my paper Hunting multiple quanta by selective least squares.

Survopoint display mode

It was a rather simple task to implement this technique (available in SURVO MM from ver.3.35). The lines in the edit field of Survo are displayed by using pointer variables of the C language. Thus when switching edit lines, no line is actually moved; only their pointers are temporarily 'updated'.

All Survopoint lines are indicated by the '~' (tilde) character in the control column. At the end of such a line a marking of type ~x must exist. x is any of the lowercase characters a,b,...,z. For any x, a line having x in its control column must exist in the same edit field (typically outside the Survo window) and this line tells how the corresponding Survopoint line is displayed.

For example, in this demo the display mode of English proverbs (appearing in the latter part of the demo) is defined as follows:

e 30 159 S
*    The road to hell is paved with good intentions.
*    He laughs best who laughs last.
*    A smooth sea never made a skilled mariner.
*    Truth is stranger than fiction.
*    A friend to all is a friend to none.
*    Be swift to hear, slow to speak.
*    Knowledge in youth is wisdom in age.
 ...

Four-dimensional cube

My lecture notes (1995) on multivariate statistical methods (in Finnish) include an appendix about multidimensional hyperspheres and hypercubes. The main purpose is to show properties of such abstract objects and give an idea how things become more complicated in higher dimensions but are still tractable.

One of the illustrations is a graph of a 4-dimensional cube represented as 2-dimensional projections. This graph resembles a draftsman's plot (scatterplot matrix) of multivariate statistical data.

Here this graph is generated by a series of Survo operations triggered by an /ACTIVATE sucro command. Below is a complete description (an extract from a Survo edit field) about how the graph has been created:

 65 *
 66 *The following sucro command activates all commands having a '+' in the
 67 *control column and thus the final graph will be automatically created:
 68 *
 69 */ACTIVATE +  (Activated commands are displayed here in red.)
 70 *It is possible to draw each 2-dimensional projection of a 4-dimensional
 71 *cube as a single line graph of edges since the degree of each vertex
 72 *is 4. Then there exists an Eulerian circuit where each edge is
 73 *traversed just once.
 74 *Consider the cube in a 4-dimensional space so that vertices have
 75 *coordinates (x_1,x_2,x_3,x_4) where each x_i is either 0 or 1.
 76 *Then the following matrix gives an Eulerian circuit in this
 77 *4-dimensional cube:
 78 *
 79 *MATRIX C4 ///
 80 *0 0 0 0
 81 *0 0 0 1
 82 *0 0 1 1
 83 *0 0 1 0
 84 *0 0 0 0
 85 *0 1 0 0
 86 *0 1 0 1
 87 *0 1 1 1
 88 *0 1 1 0
 89 *0 1 0 0
 90 *1 1 0 0
 91 *1 1 0 1
 92 *0 1 0 1
 93 *0 0 0 1
 94 *1 0 0 1
 95 *1 1 0 1
 96 *1 1 1 1
 97 *0 1 1 1
 98 *0 0 1 1
 99 *1 0 1 1
100 *1 0 0 1
101 *1 0 0 0
102 *1 1 0 0
103 *1 1 1 0
104 *0 1 1 0
105 *0 0 1 0
106 *1 0 1 0
107 *1 0 1 1
108 *1 1 1 1
109 *1 1 1 0
110 *1 0 1 0
111 *1 0 0 0
112 *0 0 0 0
113 *
114 +MAT SAVE C4
115 +MAT TRANSFORM C4 BY X#-0.5   / Centering (0,1) -> (-0.5,0.5)
116 +MAT CLABELS "X" TO C4        / Column labels X1,X2,X3,X4
117 *
118 *The regular 2-dimensional projections of this hypercube are plain
119 *squares and thus not very interesting.
120 *
121 *A better view is obtained by making an "arbitrary" 4-dimensional
122 *rotation:
123 *
124 +MAT T=ZER(4,4)
125 +MAT TRANSFORM T BY sin(31*I#*J#)  / "arbitrary" T
126 *
127 +MAT GRAM-SCHMIDT DECOMPOSITION OF T TO Q,R  / Orthogonalization of T
128 +MAT K=C4*Q                  / Rotation of the hypercube by orthogonal Q
129 +MAT CLABELS "dim" TO K      / Column labels dim1,dim2,dim3,dim4
130 *
131 *Combining the rotated and original cube into one matrix KB:
132 *
133 +MAT KB=ZER(33,8)
134 +MAT KB(1,1)=K
135 +MAT KB(1,5)=C4
136 *
137 *.......................................................................
138 *Plotting all six 2-dimensional projections separately:
139 *
140 *SIZE=1000,1000 SCALE=-1,1 HEADER= XDIV=0,1,0 YDIV=0,1,0 FRAME=3
141 *FRAMES=F F=0,0,1000,1000 PEN=[SwissB(30)]
142 *XLABEL= YLABEL= LINE=[line_type(2)][line_width(0.2)],1 TEXTS=T
143 *
144 +PLOT KB.MAT,dim1,dim2 / DEVICE=PS,A12.PS T=1_-_2,750,50
145 +PLOT KB.MAT,dim1,dim3 / DEVICE=PS,A13.PS T=1_-_3,750,50
146 +PLOT KB.MAT,dim1,dim4 / DEVICE=PS,A14.PS T=1_-_4,750,50
147 +PLOT KB.MAT,dim2,dim3 / DEVICE=PS,A23.PS T=2_-_3,750,50
148 +PLOT KB.MAT,dim2,dim4 / DEVICE=PS,A24.PS T=2_-_4,750,50
149 +PLOT KB.MAT,dim3,dim4 / DEVICE=PS,A34.PS T=3_-_4,750,50
150 *
151 *.......................................................................
152 *Plotting two opposite 3-dimensional cubes in different colors (blue and
153 *red):
154 *
155 *SIZE=1000,1000 SCALE=-1,1 HEADER= XDIV=0,1,0 YDIV=0,1,0 FRAME=0
156 *XLABEL= YLABEL=
157 *               *blue=[color(1,1,0,0)],1 *red=[color(0,1,1,0)],1
158 +PLOT KB.MAT,dim1,dim2 / DEVICE=PS,B12.PS IND=X1,-0.5 LINE=*blue
159 +PLOT KB.MAT,dim1,dim2 / DEVICE=PS,C12.PS IND=X1,0.5  LINE=*red
160 +PLOT KB.MAT,dim1,dim3 / DEVICE=PS,B13.PS IND=X1,-0.5 LINE=*blue
161 +PLOT KB.MAT,dim1,dim3 / DEVICE=PS,C13.PS IND=X1,0.5  LINE=*red
162 +PLOT KB.MAT,dim1,dim4 / DEVICE=PS,B14.PS IND=X1,-0.5 LINE=*blue
163 +PLOT KB.MAT,dim1,dim4 / DEVICE=PS,C14.PS IND=X1,0.5  LINE=*red
164 +PLOT KB.MAT,dim2,dim3 / DEVICE=PS,B23.PS IND=X1,-0.5 LINE=*blue
165 +PLOT KB.MAT,dim2,dim3 / DEVICE=PS,C23.PS IND=X1,0.5  LINE=*red
166 +PLOT KB.MAT,dim2,dim4 / DEVICE=PS,B24.PS IND=X1,-0.5 LINE=*blue
167 +PLOT KB.MAT,dim2,dim4 / DEVICE=PS,C24.PS IND=X1,0.5  LINE=*red
168 +PLOT KB.MAT,dim3,dim4 / DEVICE=PS,B34.PS IND=X1,-0.5 LINE=*blue
169 +PLOT KB.MAT,dim3,dim4 / DEVICE=PS,C34.PS IND=X1,0.5  LINE=*red
170 *
171 *Coloring the projections:
172 +EPS JOIN K12,A12,B12,C12
173 +EPS JOIN K13,A13,B13,C13
174 +EPS JOIN K14,A14,B14,C14
175 +EPS JOIN K23,A23,B23,C23
176 +EPS JOIN K24,A24,B24,C24
177 +EPS JOIN K34,A34,B34,C34
178 *
179 *Entering coordinates for projections in the final setup:
180 *K12=K12,0,2000
181 *K13=K13,0,1000   K23=K23,1000,1000
182 *K14=K14          K24=K24,1000,0      K34=K34,2000,0
183 *
184 *Combining the parts:
185 +EPS JOIN CUB4,K12,K13,K14,K23,K24,K34
186 *
187 *Creating the result
188 *as a PostScript file:
189 +PRINT CUR+1,X TO Cube4.PS
190 % 1500
191 - picture CUB4.PS,*,*,0.47,0.47
192 X
193 *Making and displaying
194 *a PDF file Cube4.PDF:
195 +/GS-PDF Cube4.PS
196 *
197 *The result is converted
198 *and displayed here
199 *as an EMF file.
200 *

In the appendix (pp. 182-3) it is also shown e.g. that the number of m-cubes in an n-cube is K(n,m)=C(n,m)2^(n-m), m=0,1,2,...,n. Thus, for example, the number of edges in a cube is C(3,1)*2^(3-1)=12 and the number of cubes in a 4-dimensional cube is C(4,3)*2^(4-3)=8. These 8 cubes are shown below as 2-dimensional projections to first two coordinate axes.

The first pair is the same as in the 1-2 plot defined on lines 157-159 in the template above and the remaining three pairs are obtained by changing X1 on lines 158 and 159 to X2,X3,X4, respectively.

The generating function of K(n,n-m) numbers is f(s)=(s+2)^n in the same way as (s+1)^n is the generating function of the binomial coefficients C(n,m). The total number of "parts": vertices (m=0), edges (m=1), faces (m=2), cubes (m=3), etc. in an n-dimensional cube is then f(1)=(2+1)^n=3^n.

'Word' processing by mouse

Copies of various items in the edit field can be made in various ways.
Traditional means are the COPY command, the key alt-F4 for rectangular blocks, and the key alt-F2 for text.

For 'words' (contiguous strings separated by blanks) the best method from version 3.37 onwards is based on two mouse-clicks:
1. Click the word to be copied by the rightmost button,
2. Select the place where to copy the word by the leftmost button.

Immediately after the first copy, more copies can be made by the
leftmost mouse button.
If the mouse is pointing at a blank space between existing 'words',
the copy is inserted between these words.
If the mouse is pointing at a 'word' (a non-blank character),
this 'word' is replaced by the copy.

3. The copying process is terminated by the DEL key.

- - - - - - - - - -

When using Survo there is no absolute need for working with a mouse. For many people, operating with a classical mouse is a nuisance causing physical stress and pain.

For many years, at last for me, a 'RollerMouse' (coupled with the splendid IBM PC/AT keyboard) has been much better as a pointing device. When using it there is no need to move hands or wrist away from the keyboard. All mouse functions can be executed by minimal moves of the fingertips.

Edges and diagonals of a regular n-sided polygon

By means of Survo and Mathematica it is shown two properties related to lengths of edges and diagonals of a regular n-sided polygon inscribed in a unit circle.

This will be a part of a study
http://www.survo.fi/papers/Polygons2013.pdf
that is currently in preparation.